Your Knowledge Base Is the Problem, Not the AI

Almost every evaluation call reaches the same objection, usually right after the demo: "Garbage in, garbage out. Our knowledge base is a graveyard - how good can your AI be if that's what it's reading?"

It's the best objection we get, and the first half is right. An AI is only as good as what it reads. The mistake is assuming the knowledge base is what it reads. Two things get said on the same call, minutes apart:

Both are true, and each causes the other: stale because nobody updates it, nobody updates it because nobody reads it, nobody reads it because it's stale. You don't break that loop by hiring a technical writer.

Your knowledge base is the summary. Your tickets are the real thing.

A KB article is one person's memory of a fix, written up once, rarely checked again. Your ticket history is what actually happened: the steps that closed the case, when they applied, the first attempt that failed, the internal note saying "don't follow the runbook here, it's wrong."

The volumes aren't close either - a few hundred articles against tens of thousands of solved tickets, growing every day, written by the people who did the work. We analyzed millions of IT tickets building Harmony: the answer to the most-asked question at a company has usually been typed out by a human hundreds of times, and never once into the KB.

Why nobody updates it: the math, not the discipline

Writing docs is the only job in the queue where the payoff arrives late, invisibly, and to somebody else. Eight minutes to fix the problem, twenty-five to write it up properly - screenshots, the conditions it applies to, the two edge cases you only know because you hit them. Nobody makes that trade at 5pm with eleven tickets open, and they are right not to.

Every knowledge program - an article per ticket, quarterly reviews, a dedicated KB owner, KCS - is an attempt to beat that math with process. They hold for about a quarter.

The reader isn't a person anymore

Employees never wanted an article. They wanted the thing done. An article is a request to go do the work yourself. Once an AI agent sits in front of the knowledge base, it becomes the main reader - and what makes an article good changes completely. Not tone, not formatting, not where it sits in the tree. Four things:

• One topic per article. The AI reads articles in pieces, so a procedure split across three pages comes back half-stitched to something else.
• Spell out who it applies to. Mac or Windows, contractor or employee, which license, which region. The agent can check all of that against your identity provider, HR system, and device management - but only if the article says it matters. It cannot read "usually."
• Real names for real things. The actual group, the actual setting, the actual approver. Not "contact the right team."
• Say what to do when a step doesn't fit. Otherwise the agent guesses instead of handing off to a human.

That is the hardest stuff to write by hand and the first thing dropped when the queue is full. It's also what your ticket notes are already full of.

What "the KB writes itself" actually takes

Not pointing an AI at your wiki and asking it to rewrite everything - that gets you better-written wrong answers. Five things have to happen:

1. Group tickets by what people actually asked

Not by the category on the ticket. That was picked from a dropdown by someone guessing, then overridden by whoever picked it up - it's the least reliable field on the record. Group by the words the employee typed. Each group of repeat questions is one article you need.

2. Pull out the fix that worked, not the whole thread

You don't want the transcript. You want the steps that closed the case, and when each one applies. Your ticket data already tells you which version to trust: the one that closed without being reopened, without a second escalation, quickly, without a bad satisfaction score. A fix that worked three hundred times beats whatever the original runbook said.

3. Write in order of demand, not opinion

Two things decide what to write next: how often people ask, and how badly your KB answers it today. You can measure the second one this week, with no new software. Take the top question from each group, type it into your existing knowledge base the way an employee would, and see whether what comes back actually answers it. Most audits skip this and start from the article list instead - which is how teams polish the VPN article a fourth time while the question behind two hundred tickets a quarter has no answer at all.

4. Judge freshness by what contradicts, not by dates

"Last reviewed" tells you nothing. A three-year-old article that's still correct is fine. A recent one that contradicts how the last forty tickets were actually solved is what produces confident wrong answers. Compare what the article claims against how the same problem got fixed lately, and when they disagree, the tickets are right - flag the article. Your ticket flow is a review cycle that runs whether anyone schedules it or not.

5. Let people check the work, not do it

Every generated article should link back to the tickets behind it, so a reviewer can see where each claim came from. That turns the human job from writing into checking - "right, except we stopped doing that for the EU entity" - two minutes instead of twenty-five. That is the only version of the trade a busy team makes week after week.

What changes

The prerequisite flips. A decent knowledge base is what you get out of running AI on your ticket history, not what you need before you start. The six-month cleanup you were told to finish first is where these projects go to die.

Deflection stops being a writing project. Search only surfaces what already exists, wrong articles included - we've written before about why that isn't enough. Creating and maintaining the content is a different job from finding it.

You can finally measure it. Three numbers instead of an article count: how much of what people ask is covered, how often an answer is backed by a real article, and how many articles disagree with recent fixes.

Knowledge stops walking out the door. The engineer who knows the one strange fix for the older laptops has already written it down forty times, in forty tickets. Right now it leaves when they do.

Broken processes get loud. When a group of tickets keeps growing and no article moves the number, the answer was never an article. The request is confusing, the process is broken, or something is misconfigured. Docs were covering for it.

The honest caveats

Ticket history is messy too: "resolved - spoke to user," closures with no notes, fixes that were wrong and reopened a week later. What saves it is repetition. You are not trusting one ticket, you are trusting the answer that shows up across hundreds, while one-off junk never repeats enough to matter. Every KB article, by contrast, is one person's word, one time.

Two things are genuinely work. Tickets contain passwords, personal data, and customer names, so that has to be stripped out before anything gets written, not cleaned up afterwards. And an article inherits the audience of the tickets it came from - an internal escalation path should not turn into a page every employee can read - so who-can-see-what has to be decided as articles are created.

And whatever isn't in your history - a tool you rolled out last week, a policy nobody has hit yet - still needs a person to write it once. Small pile. It's also the only pile that needs a person.

Ask the question about the right data

"Shitty data in, shitty data out" is exactly the right question to put to a vendor. Just point it at the right input. Your knowledge base is a summary somebody stopped maintaining. Your ticket history is the real record - dated, in context, written by whoever solved the problem, growing every day without anyone being nagged to contribute.

You've had the material for years. You just filed it under closed.