5What Can Go Wrong

The risks of using AI.

Four ways it goes wrong: hallucinations, slop, privacy, and brain rot. Two of them have already ended a career in our state’s own federal court.

Risk 1HallucinationsReal cases, fake quotes. Fake cases, perfect formatting. Risk 2SlopText that sounds finished and says nothing. Risk 3PrivacyWhat did the contract promise, and who signed it? Risk 4Brain rotThe risk that will actually get you.
Risk 1 · Hallucinations

Don’t be the headline.

A veteran Assistant U.S. Attorney in Raleigh let AI help write a brief. He didn’t check it. Within three months he was out of a job, and his name was all over the national legal press.

17
Years at the U.S. Attorney’s Office, EDNC

Not a first-year associate.

Not a solo practitioner cutting corners.

A career government lawyer, in our state’s own federal court.

Experience didn’t save him.

  1. Brief filed

    Fabricated quotations and misstated holdings, attributed to real Fourth Circuit opinions. Opposing counsel caught it.

  2. “A clerical error”

    Asked to explain, he called it a draft filed by mistake. No mention of AI.

  3. He admits using AI

    The judge puts up slides of errors in his other recent filings.

  4. Fired

    He offered to retire. DOJ fired him instead.

  5. Public reprimand

    Written order from the court, plus a DOJ OPR investigation.

“The worst decision I’ve ever made in my 30‑year career.”Rudy Renfer, at the March 10 hearing. Source: Fivehouse v. U.S. Dep’t of Defense, No. 2:25-cv-41 (E.D.N.C. Apr. 2026) (order of public reprimand); ABA Journal (Mar. 11, 2026); Bloomberg Law.
The full story · scroll-drivenDon’t Be the HeadlineThe filing, the timeline, the hearing, the headlines, and what it means for a prosecutor who carries the State’s credibility.
Four months later · same office

It happened again.

In July 2026 the court asked a different AUSA for a copy of a case cited in a motion to dismiss. She searched PACER, Westlaw, and her own files, then told the court the truth.

Her best explanation: a Google search, at a time she didn’t know search engines were serving AI-generated results with “phantom” citations. The brief was filed in 2025; the office’s Westlaw cite-check requirement didn’t start until March 2026.

Thomas v. U.S. Dep’t of Justice, No. 5:25-cv-242 (E.D.N.C.)Doc. 25 · filed 07/15/26
Bowers v. Transp. Sec. Admin., 2015 WL 3442014, at *2 (E.D. Va. May 27, 2015)
This case does not exist.
Federal Defendants’ Response to Court Order (filed July 15, 2026).
You don’t have to open a chatbot to be using AI.
Before you sign anything

Five habits. Any one of them would have prevented both cases.

  • Pull every case yourself.

    If a citation came from AI, a search engine, or a colleague, open the opinion in Westlaw, Lexis, or the court’s own site before it goes in a filing.

  • Find every quotation on the page.

    Real case, fake quote was the Renfer pattern. Search for the exact words. Confirm the pin cite exists.

  • Treat search-engine answers as AI.

    AI overviews sit on top of ordinary Google results. A starting point, never a source.

  • Run a cite-checker before filing.

    A Westlaw cite-check would have flagged the fake case. Make it a standing step, not a rescue.

  • If it happens, tell the whole truth the first time.

    “Clerical error” didn’t hold up. The hearing is what ended the career.

“Always personally verify each quote or proposition with your eyes in an actual case or law or other valid source.”U.S. Attorney’s Office, E.D.N.C., guidance to staff, as quoted in the reprimand order

Interested in more lawyers misusing AI? kylebahr.netlify.app/2000 tracks the cases.

Risk 2 · Slop

Text that sounds finished and says nothing.

Slop is the writing a model produces when you give it nothing to work with. It is easy to spot once you know the tells, and the one tell that matters for us is fabricated authority formatted perfectly.

Exhibit A · the introductionGenerated · 0.8 seconds

In today’s rapidly evolving digital landscape, AI slop has emerged as a transformative force that is reshaping the way we think about content. It’s not just text; it’s a rich tapestry of words, woven together with purpose, passion, and absolutely no information. As legal professionals navigate this complex and ever-changing terrain, it is crucial to delve into the multifaceted nature of slop. Let’s dive in! 💡Specific facts, names, dates, or citations in this paragraph: none.

Read that paragraph in your most earnest voice, then count the facts. There are none. That is the pattern: slop has no nouns you can check. Real writing names a defendant, a date, a statute, a page.

The companion page walks the ten tells, each one committing the offense it describes, with a quiz and a machine that turns a plain sentence into slop.

Interactive field guideSpot the SlopTen tells, a six-question quiz, and the slop machine.
Risk 3 · Privacy

Not “is AI safe?” What did the contract promise, and who signed it?

Which account
Free accountNever.

Free consumer terms are not sufficient to protect confidential information. I’d go so far as to say you could get disbarred using one.

Paid personal accountLegally sufficient, likely not permitted.

A paid account sufficiently protects your data, but it is still going to be against AOC policy.

Enterprise accountThe only option for case material.

Work AI should be done with an enterprise work account, through AOC or the Conference.

What to check in any tool

Read the settings, then the contract. Tick each one you have confirmed for the tool in front of you.

0 of 5 confirmed
RememberEvery door in is also a door out. Know what your tool is connected to before you put a case in it.
Risk 4 · Brain rot

The risk that will actually get you.

“Why are you using AI to do basic issue spotting work that attorneys do every day?”Magistrate Judge Robert T. Numbers II, at the Renfer show-cause hearing
Keep the reps

Don’t let it do your issue-spotting.

  • You learned to read a police report by reading thousands of them.
  • A first-year with an AI screen on every file from day one never builds that instinct.
  • The senior lawyer can use it safely; the office that skips the reps can’t tell when it’s wrong.
  • Office policy question: who gets the tool, and on what?
The balanceThe advantages are real: faster understanding of a case, far more efficient retrieval. Keep the judgment yours.