On 5 March 2024 the Information Commissioner's Office (ICO) closed a consultation on its guidance for employment practices and data protection relating to recruitment and selection. The proposed guidance includes a section on automated decision making and profiling.
What is automated decision making and profiling?
Automated decision making refers to decisions that can be made without any human involvement and are made by computer programs or Artificial Intelligence (AI). Often these decisions involve profiling, based on a candidate's personal data and data from external sources.
Profiling involves cross referencing multiple sources of data and collating it to create a profile of an individual. In the recruitment sector this might be used to make inferences about a candidate's suitability for a job.
What does the proposed ICO guidance say?
The proposed guidance advises that automated decision making that has no human involvement is restricted by law and can only be used in certain circumstances. For recruitment, in practical terms, this means that candidates cannot be shortlisted based on automated decisions. However, where there is "meaningful" human involvement, the guidance suggests that automated decision making can be used and fully automated decisions can be made with explicit consent if there is safeguarding in place, such as the opportunity to request a human intervention.
The guidance suggests that the ICO will only consider human involvement to be "meaningful" if the ultimate decision at each stage of the recruitment process rests with the human, rather than the AI. There must be checks and balances in place so that the recommendations, predictions and other AI outputs can be overridden by the human reviewing them and making the decisions.
What should we do?
There are many risks with the use of AI and automated decision making, for example if the underlying programs introduce unlawful discrimination or other unintentional bias into the recruitment process. The proposed guidance suggests that recruiters should undertake a thorough data protection impact assessment (DPIA) to assess the risks that arise from using AI and automated decision making in their recruitment process and act to mitigate them.
The ICO has not yet given any indication about when the finalised guidance will come into force. However, in preparation, recruiters need to have a good understanding of the programs they use and clear policies in place that outline what appropriate use of the programs is and the safeguards to mitigate risk. Recruiters should also consider whether the programs they are using are appropriate, including whether they are trained on appropriate data sets.
The proposed guidance suggests that the ICO will have a keen focus on the use of automated decision making in the recruitment process and recruiters should be prepared for their practices to face scrutiny.