Imagine being refused an opportunity and discovering that there is nobody you can explain yourself to. There is a result, perhaps a polite message, but no conversation in which the circumstances of your life can make a difference. You can submit the same information again. Whether anyone can reconsider what it means is another matter.
For a disabled person, that imagined situation poses a particular problem: what happens when a system interprets a different way of doing something as an inability to do it?
Digital Watch Observatory reports that UN experts have warned artificial intelligence could reinforce disability discrimination. The warning concerns a risk, rather than establishing that every use of AI produces that result. But it invites us to examine something more intimate than the technology: our willingness to let a convenient judgement become a final one.
Consider a hypothetical recruitment process. An applicant needs an adjustment to demonstrate their ability. If the process cannot accommodate that request, the employer may never discover what the applicant could contribute. The procedure has answered its own question successfully while leaving the important question untouched.
There is a reasonable objection here. Human judgement is hardly free of prejudice, and requiring everyone to plead their circumstances before a sympathetic gatekeeper is no attractive alternative. A well-designed automated process might remove some arbitrary obstacles. We should leave room for that possibility.
Even so, consistency alone cannot tell us whether a process is fair. An unsuitable requirement can be applied impeccably to every applicant.
The practical responsibility rests with whoever chooses to use the system. Before accepting its decisions, they need to establish how someone can challenge a mistake, request an adjustment and reach a person with the authority to change the outcome. An appeals address is of limited use if everything sent to it merely travels round another automated circuit.
For those making these choices at work, this is an ordinary responsibility with potentially considerable consequences. Ask what happens to the person whose circumstances do not fit. Follow their route through the process. If it ends with a refusal that nobody is empowered to reconsider, there is still work to do, however efficiently the software runs.