October 8, 2026Authority AI Editorial

Make Your Agent Check Its Sources Before You Do

A glowing glass lens hovers over a scattered stack of document cards, and three thin beams of light from separate sources meet on one card in the centre.

Ask an AI agent to research a question and it will usually come back with something tidy: a summary, a few confident claims and a neat list of references. It reads like the work of someone who checked.

Sometimes the references are real. Sometimes they are close to real, with the right author and the wrong year. And sometimes they don't exist at all. The trouble is that all three look the same on the page, so you can't tell which is which without doing the checking yourself.

A research skill moves that checking to the agent. It's a written method for research work, and it tells the agent what to verify before it hands anything back.

What happens when nobody checks

In 2023, two researchers, William Walters and Esther Wilder, measured how often a chatbot invents references. They asked ChatGPT to write short literature reviews on 42 topics, which gave them 84 papers and 636 citations to check. They searched databases and websites for every one.

With the older model, GPT-3.5, 55 percent of the citations were fabricated. With GPT-4, 18 percent were. Among the citations that pointed to real work, 43 percent of the GPT-3.5 ones and 24 percent of the GPT-4 ones had substantive errors, such as wrong authors, titles or journals. The most common mistakes were in the numbers: volume, issue and page, and the year of publication.

Tools built for one profession still get things wrong. In 2024, researchers at Stanford tested AI legal research tools on more than 200 legal questions. Lexis+ AI and Ask Practical Law AI produced incorrect information more than 17 percent of the time. Westlaw's AI-Assisted Research did so more than 34 percent of the time. The authors wrote that lawyers "may find themselves having to verify each and every proposition and citation provided by these tools."

The models will keep changing. The lesson from both studies is the same: a reference that looks right has not yet been checked.

Why asking "are you sure?" doesn't work

The obvious fix is to ask the agent whether its sources are real. A New York court case shows why that isn't enough.

In Mata v. Avianca, a lawyer filed a brief that cited court decisions found with ChatGPT. Six of them, including one called "Varghese", did not exist. Before the truth came out, the lawyer had asked the chatbot directly: "Is Varghese a real case" and "Are the other cases you provided fake". It replied that the cases were real and could be found in the standard legal databases.

In June 2023, Judge P. Kevin Castel sanctioned the lawyers. The two lawyers and their firm were ordered to pay a $5,000 penalty and to write to their client and to each judge falsely named as the author of a fake opinion. The judge was clear that the tool itself was not the problem: "there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings."

Walters and Wilder found the same pattern in their study. They report that ChatGPT "often provides inaccurate responses when asked to verify the legitimacy of the works it cites." Asking the agent to vouch for its own answer only gets the same answer twice. Real checking means going outside the answer, to the source itself.

The three habits a research skill writes down

A good research skill gives the agent three habits, and each one closes a gap the studies above found.

Every claim gets three independent sources. One source can be wrong. Three sources that agree, and that didn't copy each other, are much harder to fool. Independent is the important word. Three news stories rewriting the same press release count as one source, so the skill tells the agent to trace each one back to where the claim started.

Every reference gets checked as real. The agent opens each source it cites and confirms the details match: the author, the title, the year, the journal or publisher, and the volume and page numbers. Those numbers deserve the closest look, because the Walters and Wilder study found that is where real citations most often go wrong. If a reference can't be found, it comes out of the draft, and the agent says so.

Every claim keeps a record of where it came from. For each claim, the agent notes the source, where in the source the claim appears, and the date it was read. That record is what lets you, or anyone after you, check a single line without redoing the whole job.

What the agent hands back with a method

Here's an illustration of the difference, not a logged run. Say you ask for a short brief on how often small businesses use AI tools.

Without a research skill, you might get four confident paragraphs and five references. Two of them are real, one has the wrong year, and two point to reports nobody can find. There's no way to tell which is which without checking all five yourself.

With a research skill, the brief comes back in a different shape. Each claim has its sources beside it. One figure is marked as supported by only one source, so it's flagged rather than stated as fact. A note at the end lists a reference the agent dropped because it couldn't confirm the report existed. You might still disagree with how the agent read a source. But you can see exactly what it relied on, and you can check any line yourself.

What stays with you

A research skill doesn't remove you from the job. It changes what you spend your time on.

The agent can confirm that a paper exists and says what it claims to say. It is less reliable at judging whether that paper is any good: whether the sample was large enough, whether the publisher is reputable, whether a newer study has overturned it. That judgement is still yours, and a good skill tells the agent to flag weak or one-sided sources for you to look at, not to quietly drop them or quietly keep them.

The gatekeeping role the judge described doesn't move to the agent. What moves is the legwork. Before, you had to find every reference and check it. With the method written down, you check the agent's record and read the one or two sources that matter most.

Where to start

Your first research skill can be short. Take the checklist below, put it in your own words, and add the traps you've hit before.

Before writing, list every factual claim the answer will rest on.
For each claim, find at least three sources that don't copy each other, and trace each one back to where the claim started.
Open every source you cite. Confirm the author, title, year, publisher and page numbers against the source itself, not against your memory of it.
If a source can't be found, remove it and say so in a note at the end.
Mark any claim supported by fewer than three sources as unconfirmed.
Keep a record beside each claim: the source, where in it the claim appears, and the date it was read.
Flag any source that looks weak, old or one-sided for a person to judge.

Run your next research job with that method attached, then compare the result with what you usually get. Wherever you still had to fix something, add a line to the checklist so the agent catches it next time.

You can also start from methods other people have already written. The Research pack in the AI Ultimate Skill Collection holds 206 skills for finding, checking and citing information, including one that backs each claim with at least three separate sources and one that catches invented authors, wrong years and made-up journals. The full collection is 3,029 ready-made skills in 12 packs, one for each kind of work, such as coding, research or finance, and it works with Claude Code and other agents that use skills.