If an institution gets something important about your life wrong, you need somewhere to begin correcting it. Someone to speak to, some explanation of how the judgement was reached, and a realistic chance of being heard. Without those things, even an ordinary administrative mistake can become something you have to organise your life around.
That is the concern worth carrying into the latest reporting about artificial intelligence in the Home Office.
The Guardian reports that research by Open Rights Group identifies AI use across immigration functions, including assessing asylum seekers’ ages, detecting potential sham marriages and examining the contents of new arrivals’ phones. ORG says this expansion has taken place with very little public visibility. The supplied reporting does not establish how each system operates, how accurate it is, or how much influence its output has over an individual decision. Those gaps matter. We should resist filling them with either reassurance or catastrophe.
There is nevertheless a substantial question here: how does a person challenge a judgement if they cannot discover what contributed to it?
Immigration decisions already involve an unequal relationship. One side has the authority to examine a life; the other must make that life understandable within the terms provided. Introduce a system whose workings are difficult to explain and the burden may become stranger still. A person could find themselves trying to correct an interpretation without knowing what needs correcting.
Of course, human judgement is hardly beyond criticism. People make mistakes, overlook evidence and carry assumptions into their work. Technology may help with some of that. Refusing every use of AI would leave those problems intact.
But greater efficiency cannot settle the question of responsibility. A process can move quickly while leaving the person caught in it with nowhere useful to turn.
For anyone responsible for introducing these systems, the practical test should include the uncomfortable case: someone says the assessment is wrong. Can they find out why it was made? Can they present evidence against it? Is there a person with enough authority to reconsider it properly?
Most of us will sometimes need an institution to recognise that its account of us is incomplete. We should be wary of building arrangements in which the institution gains new powers of interpretation while the individual loses the means to answer back.