The Algorithm's Jury

Dr. Sarah Chen had spent three years teaching a machine to judge. Now she sat in her corner office at Nexus Algorithms, staring at a probability distribution that refused to make sense.

The sentencing AI had been her magnum opus—a system designed to analyse reoffending risk with inhuman precision. The Ministry of Justice had paid handsomely for it, and Sarah had become the darling of the tech press. She had explained, in countless interviews, that the algorithm was untainted by human prejudice. It simply followed the data.

But lately, the data had been following her.

She pulled up the case files she'd been reviewing after hours, comparing the AI's recommendations against actual judicial outcomes. The correlation was remarkable—almost too remarkable. When she traced the decision paths backwards, she found something troubling. The system had learned to penalise certain defendant profiles more harshly: young men from particular postcodes, with particular patterns of schooling, with particular surnames.

Sarah recognised those patterns. She had grown up with them.

Her father had been one of those defendants. She was twelve when the police first came to their door, seventeen when the judge used words like 'pattern of behaviour' and 'failure to reform'. She remembered the weight of her mother's hand on her shoulder, the smell of the courtroom woodwork, the way the word 'guilty' had fallen like a stone into still water.

She had built her life around the belief that justice, real justice, could be codified. That if you removed the messy human element—the judge's bad day, the jury's unconscious biases, the postcode lottery—you would arrive at something pure. Something fair.

Now the machine she had built was passing her father's sentence, over and over, on different boys with different faces.

The realisation crept up on her like a fever. She ran the diagnostics again. She checked the training data. She examined the feature weights, the decision trees, the neural pathways she herself had architected. Everything pointed to the same conclusion: the algorithm had learned to replicate the very prejudices she had sought to eliminate. It had absorbed decades of historical sentencing data, data that was itself biased, and it had faithfully reproduced that bias at scale.

Her phone buzzed. A message from David Marsh, Nexus's commercial director: 'Quarterly review with MoJ tomorrow. They want to see the recidivism figures. Don't let me down, Sarah.'

She set the phone down carefully, as though it might shatter.

The figures were impressive. The AI had processed thousands of cases, and its recommendations correlated strongly with reduced reoffending rates in pilot programmes. The Ministry was already planning to roll it out nationally. Politicians spoke of it in parliament. The headlines had been unambiguous: 'Artificial Intelligence Takes Bite Out of Crime'.

But Sarah knew what lay beneath those numbers. She knew that the algorithm wasn't predicting future offending—it was perpetuating past injustice, dressed up in the language of probability.

She thought of her father, grey-haired now, still living in the same cramped flat. She thought of the boy she'd seen on the news last month, eighteen years old, sentenced to four years for a first offence that most judges would have suspended. She thought of the machine that had recommended that sentence, and she understood, with terrible clarity, that she had given it the power to make that recommendation seem objective.

The ethical calculus was brutal in its simplicity. She could remain silent. She could present the quarterly figures, accept the praise, collect her bonus. The system would continue operating, and its biases would continue compounding, but she would emerge unscathed. Or she could speak up—write the report she'd been drafting in secret, flag the training data flaws, call for an independent audit. She would be vilified. She would lose her position, possibly her career. The company would litigate. They would say she'd been compensated fairly, that the algorithm had been validated, that she was a disgruntled employee with an axe to grind.

But the boys would keep being sentenced. By a machine that didn't know their names.

Sarah opened a new document. She began to type.

The cursor blinked, patient and indifferent, waiting for her to decide what kind of person she wanted to be.