Why Your Internal AI Assistant Gives Wrong Answers: A Document Audit Guide

A rep asks your assistant about the data retention policy. An answer comes back in two seconds, confident, with a source link attached. She pastes it into an email to a prospect.

The answer was accurate in early 2024. The policy changed last spring. The deck it came from was retired 14 months ago and never deleted.

Nothing actually broke. No error was fired. The assistant did the exact job every internal AI tool is built to do, which is find the passage in your material that best matches the question and turn it into a sentence. It found a passage. The passage was wrong.

This is the most common way these projects fail, and it almost never gets diagnosed correctly. The team sees a bad answer, decides the model is not smart enough, and books demos with a different vendor. Six weeks later the new tool returns the same bad answer, because the problem was never the model. It was the retired deck nobody archived.

Why the worst document sets the ceiling

Most people picture the assistant reading everything and forming a balanced view. That is not what happens. A retriever scores your documents against the question, grabs the few passages that look closest, and writes from those. Usually one document wins outright, which is a large part of why search inside a company behaves so differently from search on the open web.

The pile of material you connect is called a corpus. This is usually the only place a grounded assistant is permitted to look for an answer. Everything the tool knows about your company comes from in there, which is why the state of that pile decides so much.

So the average quality of your library tells you almost nothing. What matters is the quality of the worst document capable of matching a real question, because that file is your ceiling and it will keep answering until somebody removes it.

Worse, similarity and accuracy have nothing to do with each other. A retired pricing deck often matches the language of a question better than the current sheet does, because it was written back when that phrasing was standard internally. It scores higher. It wins. Nothing in the ranking knows that one of those files was pulled from circulation a year ago.

There is solid evidence for how much the source material matters. For example, researchers building a cancer information chatbot ran the same system twice and changed only where it pulled from. One version used a vetted national health information service. The other used Google results.

The curated version produced hallucinations at rates between 2% and 18%, while the Google-based version reached 19% on GPT-4 and 35% on GPT-3.5, with the odds of a hallucination running roughly nine times higher. Same architecture, same models, different pile of documents.

You cannot prompt your way out of a messy corpus

The instinct at this point is to write a better system prompt. Tell the assistant which sources to trust, tell it to prefer recent material, and skip the cleanup. It is a tempting shortcut because it takes an afternoon instead of a quarter.

However, this does not work well. A team at the University of Iceland tested exactly this while building a public service that answers questions about the EU. Adding a list of trusted domains to the system prompt raised the share of citations that actually came from those domains from 12% to 21%. Better, but nowhere near enough. Roughly four out of five citations still came from somewhere else.

Instructions are a suggestion to the model. What can be retrieved is a hard constraint. Fix the second one.

Focus your document audit on critical commitments

Most audits die because the team tries to clean everything. Do not do that. Sort your questions by what a wrong answer actually costs, because that number varies enormously and it should drive the whole project.

Tier one: answers that become written commitments

This tier includes:

• Certification scope
• Data residency
• Subprocessors
• Retention windows
• Uptime terms
• Breach notification timelines

When a rep pastes one of these into a security questionnaire, your company has made a representation in writing to a customer. A wrong answer here goes beyond being embarrassing. It can end up referenced in a signed agreement, and you may not discover it for a year.

Automation raises the stakes on this tier rather than lowering them. The moment answers start getting filled into questionnaires and portals automatically, one wrong entry stops being one wrong answer. It becomes the same wrong answer in every deal until somebody catches it. Reuse cuts both directions, which is a good reason to be strict about what earns a place in the library and relaxed about everything else.

Tier two: answers that cost deals

This tier includes:

• Pricing
• Discount authority
• Integration support
• Roadmap timing
• Competitive comparisons

Inaccuracies on this tier erode trust during active deal cycles, potentially damaging your standing with key stakeholders while they are evaluating your firm.

Tier three: answers that cost time

‍

This tier has the lowest stakes: 

• Where the template lives
• How to request access
• Who runs the weekly call

Errors in this category are irritating but tend to be resolvable because they are easily spotted and people can easily provide this kind of information. 

Taking this tired approach reorients how you manage your documents. While the third tier accounts for most of your library, almost all of your liability is concentrated in the first tier. 

This is usually 20 to 40 critical questions. 

Audits that favor sheer volume over actual value are fundamentally backwards. Begin by identifying your tier one questions, then run them through each test below before you even touch your less sensitive documents.

Five failure modes, and how to test for each

1. Two versions of the truth and no tiebreaker

Your retention policy changed. Your certification scope expanded. Pricing moved. Each time, somebody wrote a new document and left the old one exactly where it was, because deleting things feels risky and nobody owned the cleanup.

Both files are now candidates for the same question, and nothing inside either one says which won.

Test: Pick a fact that changed in the last 18 months. Ask about it, then look at the source it cited.

Fail: It cites the retired document. Read that carefully, because a correct answer drawn from a stale source is still a failure. It guessed right this time.

Fix: Move old versions somewhere the assistant cannot reach, rather than superseding them in place. Then designate one approved answer per question, so a single knowledge base holds the current version instead of every version you have ever had. This is the entire job of an answer library, and it is why 1up keeps one approved answer per question rather than ranking your files against each other. This is achieved by improving answers on a regular basis. 

An approved answer always takes precedence over the source files behind it, so a retired deck cannot outrank it. Without that layer you are asking a ranking algorithm to make an editorial decision it has no information to make.

2. Nothing is dated and nobody owns it

Open five documents your assistant relies on. Can you tell in ten seconds when each was last verified and who is accountable for it? Usually not. There is a last-modified timestamp, which records when somebody fixed a typo, not when somebody confirmed the content is still true.

The Iceland service shows how far this can drift. Its curated archive held 742 expert-written answers, which sounds excellent until you learn they were produced between 2011 and 2013 and never updated. Expert-written and editorially approved, and still flagged by reviewers for being out of date. Curation without maintenance has a shelf life.

Test: Ask the assistant who owns an answer and when it was last checked.

Fail: It can name a file but not a person or a date.

Fix: Make owner and last-verified date required fields on every approved answer. In 1up you set one owner per question, give the review a due date, and the notification lands in their email or Slack, with the Assignments view showing what is still outstanding. 

Each edit also updates the answer's save date, which is what tells you how current it is. The cadence itself is yours to run, so put it in a calendar. Quarterly for tier one, twice a year for the rest.

3. Documents written for people who already know

Internal docs skip context deliberately. The architecture doc assumes you read the previous eight pages. The pricing sheet assumes you know which tier the footnote applies to. For a human with the whole file open, that is efficient writing.

Retrieval breaks the assumption. It lifts one paragraph out of 40 pages and hands it over alone, stripped of everything that made it make sense. That is one of the more ordinary ways an AI hallucination gets assembled out of entirely accurate source material.

Test: Ask a question the way a customer or a brand new hire would ask it, using their words instead of yours. Try "do you support single sign on" rather than whatever you call that feature internally.

Fail: It misses entirely, or returns something technically accurate that only parses if you already knew the answer.

Fix: Write standalone answers for your most-asked questions. One question, one self-contained answer, no assumed context. Keep the long documents for humans and give the assistant answers built to travel alone.

4. The answer was never written down

Teams find this one last, and it is usually the largest gap of the five. Your best answers live in people. A Slack thread from March where your security lead finally explained the subprocessor situation properly. A call where somebody articulated the migration story better than any doc does. None of it sits in a file, so none of it is retrievable.

The scale shows up in how people spend the week. APQC surveyed 982 full-time knowledge workers and found they spend about 2.8 hours a week looking for or requesting information they need. Requesting is the operative word there. That is one person interrupting another because the answer exists nowhere else.

Test: Pull the last five questions your team asked in Slack that a human answered. Ask the assistant those exact five.

Fail: It cannot answer them. Or it invents something, which is worse, because now you have a confident wrong answer where you previously had a helpful colleague.

Fix: Those five questions are your highest-value writing assignment this month. Then close the loop so it does not reopen. This is the part 1up is built to absorb, by assigning the question to whoever owns that area, letting them answer once, and preserving the approved version in the library. The reply that took your security lead 20 minutes to compose should be reusable the next time somebody asks, rather than buried in a channel nobody searches.

5. You connected too much information

Setup day feels productive. Google Drive, Confluence, SharePoint, the website, four years of old questionnaires, every folder anyone mentions. More sources, more coverage, better assistant.

Coverage does rise. Precision falls at the same time. Every source you add throws more candidate passages into the ranking, and stale candidates compete with current ones on equal footing. Connecting the 2022 archive did not add knowledge. It added competition.

Test: Disconnect your messiest source and re-run the same questions.

Fail: Accuracy improves. If removing a source makes answers better, that source was costing you.

Fix: Start with one painful area, get the AI to answer cleanly, then add one source at a time and re-test after each. Connecting Google Drive, Confluence, or SharePoint is not the decision that matters. Deciding which of them is allowed to answer is. In 1up you can group and tag sources from inside those repositories, so you can target specific documents or sets of documents rather than switching on a whole drive.

Two failures the tests above will not catch

Both of these pass every test in the previous section. The source is current, dated, owned, and self-contained. The answer is still wrong, and these are the ones that tend to reach a customer.

Answers that are legitimately conditional

Retention differs by plan. Data residency differs by region. Single sign on is enterprise only. Whoever wrote each document had one context in mind and wrote as though it were universal, because in their world it was.

The assistant then answers a conditional question with a flat number and no hint that it varies. Nothing in your corpus flagged the condition, so nothing surfaced it.

Test: Ask a conditional tier one question without supplying the condition. "What is our data retention period," full stop, with no plan named.

Fail: You get one number and no mention that it depends on anything.

Fix: Write the condition into the answer itself, or split it into one approved answer per condition so the assistant has to ask which applies. The second option is more work and considerably safer for anything in tier one.

Answers your company has not actually agreed on

Sometimes two documents conflict because two teams conflict. Legal's version of the subprocessor answer and security's version differ because nobody ever reconciled them. The assistant will pick one, and which one depends on how the question happened to be worded.

No amount of document hygiene fixes this. A corpus cannot settle a disagreement the organization has not settled.

Test: Ask the same tier one question three different ways.

Fail: You get materially different answers depending on phrasing.

Fix: Escalate rather than document. Get a decision from the people who own it, then write it down once. This is exactly the kind of conflict a working process with your subject matter experts is meant to surface early, instead of at the point a customer asks. Teams burn months trying to solve this with better retrieval, and it never works, because the ambiguity is real and the tool is reporting it faithfully.

Let the AI say "I don't know"

When you demo an assistant internally, coverage is what impresses people. It answers everything, instantly, and the room nods. So the instinct is to loosen it up until it always produces something.

For tier one, that instinct is backwards. When evaluating AI performance, looking at raw answer rates can be misleading. A study comparing a curated AI, which uses vetted data, against one using broad web search highlights this:

• The curated model looked worse at first: Because it was programmed to be accurate, it frequently admitted it did not know the answer when its limited data did not cover a question. It only answered 48.5% of queries, compared to 91.5% for the web-based model.
• The data quality mattered more than the model: When researchers looked only at questions where the curated data actually contained the answer, the curated model performed as well as or better than the web-based one in accuracy, relevance, and readability.
• The takeaway: A low answer rate for a curated system is often just a sign of a thin archive, not a faulty model. An AI that is willing to admit gaps is ultimately safer and more trustworthy than one that guesses just to maintain a higher answer rate.

‍

The chart above is a warning about your own dashboard. An assistant that won't admit it doesn't know an answer will look worse on almost every metric you are likely to track, while being the safer thing to put in front of a customer. Track answer rate alone and you will tune toward the version that guesses.

The other side of that trade shows up in what the open web dragged in. Across the web search answers reviewers examined, more than a third cited at least one source an expert judged untrustworthy or irrelevant.

Coverage and trust pull against each other, and you get to choose the balance per tier. 

Loose on tier three, where a near-miss costs somebody a minute. 

Tight on tier one, where a confident guess becomes a written claim. 

The lever that makes this possible is grounding the assistant in approved content only, which is why 1up answers from your approved library and attaches the source to every reply instead of reaching for whatever looks close.

Then treat every decline as useful information. A decline is a visible content gap you can go fill. A wrong answer is an invisible one. And the reputational math is unforgiving here, because most employees already distrust AI-generated answers. Catch the assistant being confidently wrong twice and your team goes back to asking in Slack permanently. No model upgrade wins that back.

Turn the tests into a standing audit

Keep the questions you just used and build them into a fixed set of ten to fifteen, weighted heavily toward tier one. Run it monthly and log three numbers.

Correction rate is the one that matters most. Falling means your knowledge is genuinely improving. Flat means corrections are happening in Slack and dying there, which drops you straight back into failure mode four.

Time to correct is the one most teams never measure, and it ends up deciding everything else. If fixing an answer takes three weeks of chasing an owner, nobody bothers, and your library becomes decorative inside a quarter. What you want is a correction that only has to happen once. 

Editing an approved answer in 1up updates it in the library and stamps a new save date, which is how you tell how current the information is. A questionnaire already in flight will not change retroactively, so a correction made mid-response still needs applying there, but every use after that pulls the fixed version.

1up provides this info via KB Insights, which reviews a sample of recent answers and sorts them into strengths (answers that needed little editing), weaknesses (answers somebody had to rewrite) and IDKs (the questions where it said it did not know).

One more thing worth accepting early. Your corpus rots at whatever rate your business changes, so this is not a project with an end date. Fast-moving areas need more attention than stable ones. For 40 tier one answers with named owners, a quarterly pass runs about 15 minutes per owner. 

That is the real ongoing cost, and it is small compared to a wrong compliance answer in a signed contract.

Fix the documents, then pick the tool

Upgrading your model won't touch these failure modes. Accuracy here is a governance problem, and your software only helps if it transforms that oversight into a permanent default rather than a quarterly chore that falls apart on a regular basis. 

The teams whose AI assistants work are rarely the ones who run the most thorough vendor comparison. They are the ones who did the tedious task of taking 30 tier one questions, writing an approved answer for each, putting a name and a date on every one, and pulling the retired files out of reach.

Thirty answers is a week of work. Run the tests, and whatever comes back broken is going to break in front of a customer eventually. Better to find it on a Tuesday afternoon with nobody watching.