The first chatbot we built at a large enterprise kept its knowledge in one place: a retrieval layer over a curated set of documents. It worked. Then the next solutions arrived on the backlog, we needed part of the same knowledge — the same procedures, work instructions, process documentation.
This is where it started to break. Copied sources drift. Maintaining three versions of content is expensive, and after a while nobody can say which one is true. So we went looking for a better way.
The question stopped being about chatbots. Work instructions, training material, SOPs, process documentation: content sitting in SharePoint folders and in documentation scattered across the enterprise. How does any of it become accessible and useful more than once?
Most of the AI conversation runs on capability: which model, what the demos show, how close agents are to performing real work. After the second chatbot, I think the more urgent question for enterprises sits somewhere less glamorous. It is less often about whether the model is capable enough. It is whether the organisation is legible enough.
Fig. 1 — The second is a diagnosis of the organisation.
With one GenAI solution, you can still build around the mess. You collect the documents, create a dedicated knowledge base, connect a retrieval layer, tune the prompts. It works for that one context.
Then the next solution appears, and the same knowledge is needed again. A procedure used in one assistant is also relevant in another workflow. A policy explanation used by customer service may later become part of an agentic escalation path.
Here organisations face a choice: copy the knowledge into another solution, accept another silo, and move fast for now, or treat the repeated need as a signal that something should become reusable.
If every AI initiative creates its own private version of enterprise knowledge, the organisation does not become smarter. It becomes more fragmented.
Fig. 2 — Every new use case adds another place where the same content is copied, slightly changed, and slowly drifts from the source.
The scale of this is now measured. A widely cited MIT study of enterprise AI found that 95 percent of generative-AI pilots produced no measurable return. The headline got read as a verdict on the technology. Read the detail and the failures trace to brittle workflows, weak contextual learning, and organisations unable to supply structured context to the systems they were buying. The models were mostly fine. The organisations were illegible to them.
A useful operational data product is more than rows in a table. It carries the things that let a second, third, or fourth solution trust it without rebuilding it.
Fig. 3 — The unit of reuse is not the file. It is the meaning that travels with it.
The interface element, at least, now has an industry standard forming around it: a shared protocol for how systems reach context. That makes the rest of the list more urgent. A stable pipe to meaningless content is just faster fragmentation.
Most enterprise knowledge was never designed to be used by AI. It was designed for people who already knew what was missing.
It lives everywhere, and nowhere in particular — partly maintained, partly duplicated, partly outdated, and partly assumed.
Fig. 5 — The last source is the one no system can index — and the one the organisation quietly runs on.
One team knows which document is the real one. A field engineer knows the exception. A senior leader remembers why a decision was made three years ago, and that reasoning is nowhere in the system.
The organisation runs on this tacit knowledge. AI cannot use what it cannot see.
That is why I do not see this as only a data problem. It is a visibility problem and an ownership problem, and therefore a design problem.
People translate between systems every day — between policy and practice, between official process and lived reality, between what the system says and what the work requires. They know which source to trust, which field to ignore, which person to ask, which exception matters, and which rule is technically correct but operationally misleading. Most of that work is not documented. It is called experience.
If we do not make it visible, AI works with a flattened version of the organisation. It sees the document, but not the judgement. The record, but not the exception. The procedure, but not the reason people work around it. The data, but not always the meaning.
And then the answer comes back fluent, confident, and wrong in a very organisational way — wrong the way an outdated procedure is wrong. If the procedure is outdated, AI scales the outdated procedure. If ownership is unclear, AI exposes the confusion. If the same rule exists in three places, AI does not know which one reflects reality.
This is the risk I think we underestimate, and also the opportunity. Put a capable system into a workflow and it quietly surfaces everything the workflow was holding together informally.
Fig. 6 — The system does not create these gaps. It makes them legible — sometimes for the first time.
For IT and product leaders, this changes the portfolio question. Alongside “which AI use cases should we build” sits a second question: which repeated knowledge needs should become shared capabilities? A repeated need is rarely a content problem. It is usually a capability trying to appear.
Fig. 7 — Read the repetition as a signal. The duplicated effort is the shape of a missing shared capability.
This is where service design earns its place in enterprise AI: it makes the work visible enough to architect well. It maps where knowledge is created, where it moves, where it gets stuck, where context disappears, and where handovers become fragile. It reveals the difference between the process on paper and the process in use, and it translates between business intent and technical implementation.
That translation matters because AI systems need the right information, in the right context, with the right meaning, at the right moment. And that bar rises sharply as we move from assistive AI into more agentic patterns.
Retrieve, answer, repeat. The context it needs is mostly the question in front of it.
- Know what it is allowed to do
- Discover which capabilities exist
- Retrieve the right context, not just text
- Understand which source is trusted
- Call tools and preserve memory
- Escalate to a human when accountability requires it
Fig. 8 — The jump from assistive to agentic is a jump in how much organisational context has to be made explicit.
Policies become constraints that systems can call. Procedures become workflows that can be interpreted and improved. Data becomes a capability that can be discovered and reused. Tacit knowledge becomes something we capture, test, and maintain, with people in the loop.
The industry has recently caught up to this with a name: context engineering, the discipline of getting the right knowledge in front of a system at the right moment. The finding that travels with it is that most agent failures are context failures.
Multi-agent systems make the same point at architecture level. People describe them as a set of specialised agents: one plans, one retrieves, one reviews. But organisations already work like this, with distributed expertise, handover moments, escalation paths, decision rights. The technical question is how agents coordinate. The organisational question is what they are coordinating around. If the underlying knowledge is fragmented, the agents inherit the fragmentation. If context is not designed, every handover becomes fragile.
So the work is to make the enterprise more legible, more reusable, and more capable. Before something can be reused, it has to be understood, and the questions that surface it are design questions as much as strategy ones.
Fig. 9 — Once an organisation moves from one AI solution to many, reuse stops being a build detail and becomes a leadership concern.
With reusable operational capabilities, a maintenance agent can combine telemetry, repair history, procedures, and incident classifications, drawn from sources the organisation actually trusts. The same foundations serve the customer assistant and the planning workflow. That is the real foundation beneath enterprise AI.
The organisations that benefit most from AI may not be the ones with the most impressive demos or the largest models. They may be the ones that make their own reality most usable.
Fig. 10 — This starts with mapping, ownership, handovers, tacit knowledge, repeated needs, and shared capabilities.
Because the question is no longer only, “Which model should we use?” The better question is: what does our organisation need to understand about itself before AI can safely and usefully act within it?
If you want to test where you stand, take your two most recent AI initiatives and write down what knowledge both of them needed. That list is your first shared capability, and writing it usually takes twenty minutes.
That is the enterprise AI worth building: AI as the reason an organisation finally becomes more understandable, more capable, and more humane to work in.
Sources: MIT NANDA — State of AI in Business 2025, Fortune — coverage of the MIT report, LangChain — Context Engineering for Agents, ML6 — MCP and AI agents