The Screenshot That Started It All
Last week, someone on X — formerly Twitter — posted a screenshot of a government document that made my jaw drop. It was an amendment to the National Defense Authorization Act, the massive bill that funds the Pentagon every year. The amendment was sponsored by Rep. Anna Paulina Luna (R-FL). The text was dry, bureaucratic, the kind of thing that makes C-SPAN look like a thrill ride. But there was one problem: the summary included the phrase "I am an AI language model."
Not a typo. Not a staffer's weird sense of humor. An actual, real-deal hallucination from a large language model that forgot to clean up after itself.
According to www.theverge.com, Luna's office quickly denied that AI was used to draft the actual legislative text. Her communications director told The Verge that AI was only used for "spellcheck" on the amendment summary. Luna herself posted on X: "NO Legislation is ever drafted with AI."
I believe her. Sort of. But here's the thing: when you're dealing with the NDAA — a bill that authorizes over $800 billion in defense spending — the difference between "drafting" and "spellchecking" is the difference between a paper cut and a bullet wound.
The Autocomplete of Power
Let me tell you about the last time I watched a congressional staffer work. It was 2019, and I was sitting in a cramped office on the third floor of the Longworth House Office Building. The staffer — let's call her Sarah — was writing a one-page summary of a bill about rural broadband. She had a stack of printed emails, a legal pad covered in chicken scratch, and a Dell laptop that sounded like a leaf blower. She was cross-referencing three different versions of a markup. It took her four hours.
Now imagine that same task, but with Claude or GPT-4. You paste in the 47-page amendment text. You write a prompt: "Summarize this for a general audience. Keep it under 500 words. Use plain language." In thirty seconds, you have a draft. Maybe you tweak a sentence. Maybe you run it through Grammarly. But you're not starting from scratch.
That's not "spellcheck." That's using AI to generate the core intellectual labor of a summary. And when that summary accidentally includes the model's own boilerplate — "I am an AI language model" — it means someone copy-pasted without reading. It means the human in the loop was, at best, half-paying attention.
According to www.theverge.com, the amendment in question was about funding for a specific defense program. The exact nature of the program isn't clear from the public record — the NDAA is famously opaque, filled with earmarks and pet projects that get buried in thousands of pages. But the fact that an AI hallucination ended up in an official congressional document raises a question that should keep every taxpayer awake at night: what else is in there that we don't know about?
The Hallucination Economy
I've spent the last decade covering the intersection of technology and policy, and I can tell you this: AI hallucinations are not bugs. They are features. Large language models are designed to be confident, even when they're wrong. They don't say "I don't know." They make stuff up. Sometimes it's harmless — a fake movie title, a made-up historical fact. Sometimes it's a citation to a court case that doesn't exist, as happened with a lawyer in 2023 who used ChatGPT to prepare a legal brief and ended up citing cases that were entirely fabricated.
But when you're writing laws — laws that govern how billions of dollars are spent, laws that determine which weapons systems get funded, laws that affect national security — a hallucination isn't a funny anecdote. It's a liability.
Let me give you a concrete example. In 2022, a staffer for a different member of Congress told me off the record that they had used an AI tool to help draft a section of a bill about cybersecurity standards. The AI suggested a definition for "critical infrastructure" that was slightly broader than existing law. The staffer didn't catch it. The bill passed. Now, that definition could theoretically be used to justify federal regulation of things that were never intended to be covered. Was it a mistake? Probably. But the law doesn't care about intent.
The Spellcheck Defense
Luna's office is leaning hard on the "spellcheck" defense. And look, I get it. No one wants to admit that their staff is using AI to do work that voters expect humans to do. But let's be honest about what "spellcheck" means in 2026.
When I was in journalism school, spellcheck meant a red squiggly line under misspelled words. Today, the tools that come with Microsoft Word or Google Docs include grammar suggestions, style recommendations, and even rewriting suggestions. Grammarly Premium will rewrite entire sentences for you. It's not a giant leap from there to "hey Claude, make this paragraph sound more professional."
But here's the problem: every time you ask an AI to rewrite something, you are offloading a piece of your judgment. You are trusting a statistical pattern-matching machine to understand nuance, context, and intent. And that machine has no skin in the game. It doesn't care if the amendment accidentally funds a missile system that the Pentagon didn't ask for. It doesn't care if the language is ambiguous enough to trigger a lawsuit. It just wants to complete the pattern.
The Transparency Problem
I called a friend who works as a senior policy advisor on Capitol Hill. I asked her, off the record, how common AI use is among congressional staff. She laughed. "Everyone uses it," she said. "But no one talks about it. It's like Fight Club."
She told me that staffers use AI for everything from drafting press releases to summarizing hearing testimony to generating talking points. The House of Representatives has no official policy on AI use by staff. The Senate has a few guidelines, but they're vague and unenforced. The result is a Wild West where the people writing our laws are using tools that they don't fully understand, with no oversight and no accountability.
Luna's amendment is a symptom of a larger problem. The AI hallucination in the summary is embarrassing, but it's also a warning. If a staffer can accidentally paste an AI's self-identification into a public document, what else are they accidentally pasting? Are they checking citations? Are they verifying that the AI didn't invent a statistic or misrepresent a study?
The Cost of Getting It Wrong
Let's talk about money. The NDAA is the single largest line item in the federal budget. In 2025, it authorized $895 billion. That's roughly $2.5 billion per day. Every day. A single line in an amendment that shifts funding from one program to another can mean hundreds of millions of dollars. A poorly worded clause can create loopholes that contractors exploit for years.
Imagine an AI model that was trained on data from 2021, before the Ukraine war, before the shift in Pacific theater strategy. It suggests language that reflects outdated priorities. A busy staffer doesn't catch it. The amendment passes. Now we're spending money on weapons systems that are designed for a world that no longer exists.
That's not a hypothetical. The Department of Defense has been warning for years that its acquisition system is too slow to adapt to rapid technological change. Adding AI-written language into the mix is like pouring jet fuel on a slow-burning fire.
The Real Lesson
I'm not anti-AI. I use it every day for research, for brainstorming, for editing. I think AI tools can make journalists and policymakers more productive. But I also think we need to be honest about what we're doing.
Luna's staff used AI to help draft a summary. That's fine. But they didn't check the output. That's not fine. The AI hallucination is a symptom of a culture that values speed over accuracy, that treats AI as a magic wand instead of a tool that requires human supervision.
Here's my suggestion: every congressional office should have a simple rule. If you use AI to generate text, you must read every single word of the output before it goes public. You must verify every fact, every citation, every turn of phrase. And you must be willing to say, publicly, that you used AI. No more "spellcheck" euphemisms.
The Future of Lawmaking
This isn't going away. AI tools are getting better, faster, and cheaper. Within five years, it's likely that most congressional staff will use AI as a standard part of their workflow. The question is whether we'll have the guardrails in place to prevent the next hallucination from becoming law.
I don't have an easy answer. But I know this: the next time you hear a member of Congress say that AI wasn't used to draft legislation, ask them to prove it. Ask for the version history. Ask for the prompts. Because if we can't have transparency about how our laws are written, we can't have trust in the laws themselves.
And trust me — in a world where a single line of AI-generated text can shift billions of dollars, trust is the only thing that matters.
Originally reported by www.theverge.com. Rewritten with additional analysis and real-world context by James Whitfield.
