You’ve Been Scored
California wants a human in the loop when AI fires you. By the time that law passes, the loop will have disappeared.
Robert Musil spent twenty years writing a novel he never finished, and I think that might have been the point. The Man Without Qualities (all 1,700 published pages of the fragment) is set in the Austro-Hungarian Empire during its final bureaucratic paralysis. The empire does not collapse in Musil’s telling. It administers. Every question is referred to a committee. Every committee defers to another. The parallel campaign at the novel’s center, a vast national project meant to celebrate the Emperor’s jubilee, spends three years in meetings without ever agreeing on what it is celebrating. The genius of the thing is that no one is failing. Everyone is doing their job. The system produces nothing, and nobody is responsible for the nothing.
I thought about Musil last January when the Eightfold lawsuit was filed.
Here is what happened, according to the class action brought by Towards Justice’s AI in the Workplace Accountability Project, with former EEOC chair Jenny Yang of Outten & Golden LLP representing the plaintiffs: Eightfold AI, a hiring platform used by companies including Microsoft and PayPal, scraped the personal data of over one billion workers. Social media profiles. Location data. Internet activity. Tracking pixels. From all of this it assembled a “Match Score,” a zero to five rating for each candidate. Lower-scored candidates were filtered out before any human being ever looked at their resume. The applicants were never told their data was being compiled. They were never given copies of the reports. They were never offered a chance to dispute errors.
The lawsuit does not claim the algorithm was biased. That is the part that stops me cold. The machine existed in secret, operating on a billion careers, and not a single person knew they had been scored. That was the violation.
Musil would have recognized this immediately. The empire does not need a villain. It just needs a procedure.
California is trying to fix this. In February 2026, State Senator Jerry McNerney introduced SB 947, the No Robo Bosses Act, legislation that would bar employers from relying solely on automated decision-making systems to fire or discipline workers. It would require human oversight when AI assists in termination decisions. It would give workers the right to appeal. The bill is careful and technically improved from the version Governor Newsom vetoed in 2025, when he complained that the original language placed “overly broad restrictions” on how employers could use automated decision systems.
SB 947 is well-designed. It is also solving the wrong problem.
The debate the law is having (should a human be required to review an AI recommendation before a company acts on it?) is a 2025 debate. I say this as someone who argued for human accountability long before it became a legislative agenda. When I wrote about algorithmic management a couple of years ago, the concern was a system that surveils and scores but nominally leaves humans in charge. The “human in the loop” was the proposed answer then. But the loop has been reorganizing itself. Gartner published research last August showing that 40% of enterprise applications will feature task-specific AI agents by end of 2026, up from less than 5% in 2025. A separate Gartner forecast puts 15% of everyday workplace decisions made autonomously by AI agents by 2028. Autonomously. Made.
The language is everything here. Yuval Noah Harari said at Davos in January 2026 that “when AI controls language, it shapes society.” He was talking about deepfakes and political manipulation, but the more immediate example is playing out in California’s courtrooms right now. The legal battle over Eightfold has generated two competing theories now in direct collision. Workday, facing its own discrimination lawsuit, argued that the AI vendor is an “agent,” legally liable for the discriminatory outcomes its tool produces. Eightfold’s lawyers argued the company is a “consumer reporting agency,” governed by credit reporting transparency law rather than employment discrimination law. One framing means the tool-maker owns the harm. The other means the tool-maker owes you a disclosure. Neither answers who is responsible when an autonomous agent, rather than a tool, makes the call entirely on its own.
This is where we need to spend some time with Hannah Arendt.
Arendt spent most of her adult life trying to understand how catastrophe is actually produced: through ordinary people in ordinary institutions. Born in Hanover in 1906, she fled Germany as the Nazis came to power. A French detention camp held her briefly before she made her way to New York, where she became one of the twentieth century’s most formidable political thinkers. In 1961, she covered the trial of Adolf Eichmann for The New Yorker. Eichmann was the senior Nazi official responsible for the logistics of the Holocaust: the train schedules, the bureaucratic paperwork. The operational architecture of mass murder. Arendt arrived in Jerusalem expecting a monster. She found a bureaucrat. A man who used clichés when pressed for explanations. A man who had been, within the terms of his institution, a good employee. Her resulting book, Eichmann in Jerusalem (1963), introduced the concept she called “the banality of evil,” a phrase so thoroughly misquoted over the decades that its original precision has nearly vanished.
The “banality of evil” names something precise: catastrophic harm travels through procedure. It is executed by people who follow instructions and fill out forms, people who, within the terms of their institution, perform their jobs correctly. The architecture of the system distributes responsibility so finely that no single node in the chain can feel its weight. That is where the harm lives.
An AI agent that autonomously scores and filters workers is Arendt’s bureaucracy rendered as software. Nobody at Eightfold sat down and decided which billion careers to undermine. Somebody wrote the scraping logic. Somebody calibrated the scoring model. Somebody sold the API. Somebody bought it without reading the terms. Somebody deployed it. And none of them, individually, did anything most people would call wrong.
For the companies involved, that diffuse structure is a feature. Distributed responsibility means distributed liability, which in practice means liability absorbed by no one.
There is a second philosopher worth meeting here, one much less famous than Arendt and considerably more useful for this specific problem. Philip Pettit is an Irish political philosopher who has taught at Princeton since 2002, developing what he calls “republican” political theory (the classical tradition reaching back to Rome, distinct from the American party of the same name). His central claim: freedom means the absence of domination, something categorically different from the mere absence of interference. His key distinction is deceptively simple: you can be dominated even if the dominating power never actually intervenes against you. A slave who happens to have a kind master is still a slave. The condition of domination is the existence of the capacity to harm, exercised arbitrarily, with no meaningful avenue to contest it.
Pettit spent years building out this distinction across books including Republicanism (1997) and A Theory of Freedom (2001), mostly in conversation with liberal political philosophy, which he thought confused being free with merely being left alone. The real threat to freedom, in his framework, is being subject to a power that could interfere at any moment, by criteria you cannot know, with no real recourse.
The Eightfold algorithm dominated over a billion workers who never received an adverse decision. It had the capacity to determine their futures. It exercised that capacity in secret. They had no ability to contest it. Pettit would say that every one of those billion people was unfree, regardless of whether the machine gave them a four or a zero.
This is the philosophical gap that SB 947 does not close. The bill focuses on notification and human review at the moment of termination or discipline. Pettit’s framework says the domination begins the moment the system is deployed, long before any individual harm occurs. You were dominated the day the algorithm was trained on your data without your knowledge. Human oversight of the final output does not address the condition of being subject to an opaque and powerful system in the first place.
Colorado’s AI Act takes effect June 30 of this year, 65 days from now. It is the most comprehensive state AI workplace law in the country, requiring annual impact assessments on high-risk AI employment systems and documented appeal processes for any adverse decision. Penalties reach $20,000 per violation. Illinois and Texas have their own narrower versions already active. At the federal level, Senators Hawley and Warner have proposed the AI-Related Job Impacts Clarity Act, requiring quarterly reporting on automated job eliminations. President Trump’s December 2025 executive order directed the federal government to review state AI laws deemed “inconsistent” with the administration’s plans for a national framework whose details remain unclear.
What you have, as of this month, is workers in different states with radically different protections against the same algorithms. I want to be direct: these laws are doing necessary work. I argued in this newsletter, when writing about who pays when the algorithm screws up, that the accountability question would define this era. These laws are trying to answer it honestly. They are just running at 2025 speed.
The shift that makes this conversation different from every prior version is the arrival of genuinely agentic AI: systems that receive a goal, decompose it into steps, execute across multiple tools and databases, and report back on outcomes. The gap between these systems and the AI tools of 2024 is categorical. By late 2026, according to Deloitte’s State of AI in the Enterprise survey, 97% of executives report their organization has already deployed AI agents in some capacity. What Gartner is tracking is the next step: by 2028, 15% of day-to-day workplace decisions will be made by these agents with no human in the decisional path at all. The “human in the loop” that SB 947 is trying to mandate is structurally incompatible with how these systems are being architected. The loop is gone. There is a pipeline, and the human is an optional callback at the end.
The trajectory is clear. Companies building agentic AI are making a sound bet on where work is going. The productivity case is real. The competitive pressure is real. The question worth arguing about is who designs what the agent is allowed to do.
I wrote about the Validator Class at the start of this year as a warning: the highly educated worker whose job has been reduced to clicking “approve” on AI outputs is sitting at the end of a dead-end road. The class that replaces the Validator Class is the one I find genuinely hopeful. Call them accountability architects. The people who define what the agent optimizes for. Who set its constraints and designed the escalation paths. Who signed their name, legally and professionally, to the system’s decision-making parameters. This is the hardest technical and legal work the knowledge economy has yet produced, and it is just beginning to have a labor market.
The humans who shape the agentic transition are the ones who wrote the design brief before deployment. In an economy where the agent acts, the durable human role is the one that happens before the agent ever touches someone’s career: setting the values and accepting liability for the system’s outputs.
California wants a human in the room when the algorithm fires you. That is a worthy aspiration. The problem is that by 2027 or 2028, the decision will have been made before anyone entered a room. The agent will have processed the performance data, weighted it against the team budget, cross-referenced it against the workforce plan, and surfaced a recommendation that a human will technically “review” in roughly the same way a person driving on autopilot “monitors” the road
The Musil insight is this: the empire is procedurally correct. It functions as designed. The Austro-Hungarian bureaucracy never had a villain. Neither does Eightfold. Neither, when the agents are fully deployed, will most of the organizations that use them.
The answer is to stop trying to insert humans into a process designed to exclude them and start demanding that humans take explicit, legally legible ownership of the design of that process. The standard should read: a human certified the parameters. A human warranted that the training data was lawfully obtained. A human accepted liability for the scoring model’s outputs. A human name on the architecture.
That is the job of this era. The question is whether we are training for it.