An assistant that knows your company: what it takes to build one

An internal AI assistant that answers from company knowledge is the most requested AI project we see, and the most quietly abandoned. The lifecycle is depressingly consistent: a pilot wired to a SharePoint export, two weeks of enthusiasm, then usage decaying toward zero because the answers were stale, uncited, or revealed something they should not have. None of that is a model problem. Building an internal AI assistant on company knowledge is four problems, and the model is not on the list: permissions, citations, freshness, adoption.

All four are solvable with known engineering. The trouble is that generic "chat with your documents" products solve none of them well, which is why the SharePoint chatbot disappoints so reliably.

Why the generic internal AI assistant disappoints

Indexing everything means retrieving indiscriminately. A live document store contains six versions of every policy, drafts, dead projects and outright contradictions, and a system with no notion of authority retrieves from all of it with equal confidence. The result is fluent, plausible, wrong answers, which are worse than no answer because people act on them. The retrieval mechanism itself, RAG in plain business terms, is sound. What fails is pointing it at an uncurated pile and expecting the pile to organize itself.

Permissions are the hard part

One rule, no exceptions: the assistant must never answer from a document the asker cannot open. It sounds obvious, and it disqualifies half the products on the market. An assistant indexed on everything will happily summarize the salary review file or the acquisition memo for anyone who phrases the question well. That is not a hypothetical risk; retrieval is precisely a machine for surfacing what a keyword search would have missed.

Permission-aware retrieval means the system checks the asker’s actual rights in the source systems at question time, with groups, role changes and departures included. This requirement reshapes the architecture, which is why it has to be designed in first, not patched in after the pilot has leaked something interesting.

Citations or nothing

Every answer should show the documents it came from, linked. Citations let people verify before acting, turn wrong answers into document corrections instead of rumors, and change the assistant’s posture from oracle to librarian, which is the honest posture for a system that can be wrong. They are also the core of keeping wrong answers under control by architecture rather than by hope. One further requirement: "I could not find this in our documentation" must be an acceptable answer. An assistant that always answers is lying some of the time.

Freshness, or answering from last year

Knowledge assistants age like bread, not wine. The export-based pilot is stale on day one; the production version needs continuous sync from the systems where documents actually live, a decision about which source is authoritative for each topic, and a way to retire documents that no longer apply. Curation beats coverage every time: an assistant that covers the 300 documents people genuinely need, kept current, beats one indexed on 300,000. Start with one domain, HR policies, IT how-tos or sales collateral, whichever generates the most repeated questions today, and expand only once the first domain holds.

Adoption is a product problem

The best-engineered assistant dies if it is a separate website nobody visits. It should live where people already work, Slack, Teams or the intranet search box, answer in seconds, and launch seeded with the hundred questions the helpdesk and HR actually receive, so the first week impresses instead of embarrassing. Then someone must own it: the unanswered-questions log is a to-do list for the knowledge base, and reading it weekly is the job. Measure weekly active users and answer rate, and treat decay as a defect to investigate rather than a verdict to accept.

It is fair to ask whether an off-the-shelf enterprise assistant gets you all this. It raises individual productivity from day one, and it knows nothing about your permission model, your sources of truth or your six versions of the travel policy. That difference is the entire project.

Scoped honestly, one domain, permissions first, citations always, an owner named, an internal assistant is among the highest-value systems a mid-size company can run. It is also a build we take on regularly, and the first weeks exist precisely to prove answer quality on your own documents before you commit to a rollout.