Knowledge Base Auto-Generation
Automatically identifies knowledge gaps from resolved tickets and generates draft KB articles for review.
- Continuously monitors resolved tickets for recurring issues with no existing KB coverage
- Detects outdated articles where documented steps no longer match current behavior
- Identifies the highest-impact gaps based on ticket volume and recurrence
- Drafts new knowledge articles and routes them for human review
- Flags existing articles with inaccurate or stale information
- Suggests improvements to article structure, clarity, and searchability
- Captures resolution patterns from top-performing agents to inform new content
- Tracks article performance to surface what's working and what needs refinement
- Publishes approved updates to the KB and notifies the team
What Knowledge Base Auto-Generation does
Knowledge Base Auto-Generation is Harmony's pipeline for turning resolved helpdesk tickets into new knowledge base content automatically. It mines closed conversations for recurring issues, clusters the ones that share a resolution pattern, and uses an LLM to draft a concrete, step-by-step resolution article for each pattern - classifying how automatable it is along the way. Every draft lands in a review queue rather than going live on its own, so IT admins stay in control of what actually reaches the knowledge base. The result is a KB that grows from real resolution history instead of relying entirely on manual authoring.
Who Knowledge Base Auto-Generation Is For
IT admins and knowledge base owners responsible for keeping the organization's KB current and useful.
Writing runbooks and KB articles for every recurring issue is slow, inconsistent, and easy to fall behind on; teams rarely have time to systematically mine their own ticket history for patterns worth documenting. This pipeline does that mining continuously and hands admins a ready-to-review draft instead of a blank page.
How Knowledge Base Auto-Generation Works
A scheduled internal endpoint (/internal/scheduler/trigger) discovers tenants with enough resolved-ticket volume via Athena (default: at least 20 distinct resolved conversations in the trailing window) and enqueues an analysis run per eligible tenant, respecting a cooldown period so the same tenant isn't reprocessed too often.
- Data collection
Fetches resolved/closed conversations (or, optionally, historic ITSM ticket classifications) enriched with LLM-generated category, intent, and resolution insights from the Harmony data lake via Athena.
- Clustering
Groups conversations by shared entity, then runs LLM-based initial clustering and a merge step (
run_initial_clustering,run_merge_clusters) to combine overlapping clusters above a similarity threshold; only clusters with at least 3 conversations proceed. - Article generation
For each cluster, Bedrock Claude (Opus 4.5 by default) infers the dominant resolution workflow and writes concrete steps naming who performs each one (agent, employee, it_operator, or approver), then classifies the cluster as automatable, deflectable, automatable_with_approval, structured_escalation, or junk.
- Quality evaluation
A second LLM-as-judge pass (
run_evaluate) scores classification accuracy, step specificity, and evidence grounding, flags any employee self-service ("deflection") already visible in the conversations, and assigns an overall verdict.
Results are stored as DRAFT recommendations with full evidence (source conversation/ticket IDs). An admin reviews, edits, or approves them via the recommendations API; approval publishes the article to S3, registers it in the tenant's KB catalog, and triggers Bedrock ingestion so it's searchable right away.
Capabilities
- Mine resolved tickets for recurring patterns - Queries Athena (
transform.conversations,ingest.tickets,analysis.conversation_enrichments) for closed/resolved tickets sharing a resolution pattern within a configurable window (default 90 days). - Cluster similar issues automatically - Two-stage LLM clustering groups conversations by shared intent and resolution, merging overlapping clusters above a configurable overlap threshold (0.25).
- Classify automation potential - Labels each pattern as automatable, deflectable, automatable_with_approval, structured_escalation, or junk (dropped), so teams know what to automate, document, or route rather than treating every ticket the same.
- Write grounded, step-by-step resolutions with named actors - Steps are explicitly attributed to an actor (agent/employee/it_operator/approver) and are required to be grounded in the source conversations, not invented.
- Score draft quality with an LLM judge - A second pass rates classification accuracy, step specificity, evidence grounding, and gives a high_value/useful/marginal/low_value verdict before a human ever sees it.
- Surface self-service ("deflection") signal - Flags conversations where the employee already resolved the issue mid-chat, feeding a self-serviceable rating on each draft.
- Human-in-the-loop review and publishing - Drafts start as DRAFT and require explicit (or bulk) approval before becoming live content; admins can edit content or dismiss with a reason.
- Auto-publish on approval - Approved articles are rendered to markdown, uploaded to S3, registered in the KB catalog, and ingested into a dedicated Bedrock knowledge base.
- Tenant auto-discovery with cooldown - Automatically finds tenants with enough ticket volume to justify a run and avoids re-analyzing the same data too soon.
Main use cases
Turning a recurring technical issue into a documented fix - Dozens of resolved conversations about VPN drops on network switches get clustered together; the pipeline drafts a concrete resolution workflow with steps and actors, an admin reviews and approves it, and it becomes searchable KB content the next employee with the same problem can be served automatically.
Documenting a self-service pattern nobody wrote down - A cluster of conversations shows employees already fixing a setting themselves mid-chat before a ticket was needed; the pipeline classifies it "deflectable," flags the deflection attempt during evaluation, and produces a how-to draft aimed at reducing future ticket volume for that issue.
Filtering out noise before it reaches a human - Tickets whose only "resolution" is "ticket created, someone followed up" with no described action get classified "junk" by the LLM and dropped before the review queue, keeping admin attention on patterns that are actually actionable.
Weekly tenant-wide sweep with no manual trigger - On a schedule, the pipeline checks every tenant against a minimum resolved-conversation threshold and cooldown window, then automatically kicks off a mining run for each one that qualifies.
Integrations
| Integration | Role in the agent flow |
|---|---|
| AWS Athena / Harmony data lake | Source of resolved conversations, tickets, and their LLM-generated enrichments (category, intent, resolution insights) used to build clusters |
| AWS Bedrock (Claude, Opus 4.5 / Sonnet 4.5) | Powers clustering, article generation, and the LLM-as-judge evaluation step |
| Amazon SQS | Queues per-tenant analysis run messages for asynchronous processing by the worker |
| Amazon S3 | Stores published article markdown plus a separate "intent" file used for retrieval embedding |
| Dedicated Bedrock Knowledge Base | Approved articles are ingested here (separate from the primary KB) so they can be retrieved as "insights" during live conversations |
| kb-data-api (Harmony KB Management) | Approved articles are registered as a "kb-gen" source in the tenant's KB catalog, alongside Confluence, Notion, SharePoint, and other synced sources |
| ITSM historic ticket classifications | Alternate, per-run-selectable data source in place of live conversations, not confirmed as tenant-configurable in the UI |
FAQ
It's a backend analysis pipeline (the insights-engine service), not a conversational agent. It has no chat surface; it runs on a schedule per tenant and produces draft KB articles that appear in an admin review queue.
Meet more Agents
