The Real Challenge of Multi-Agent Systems Is Information Governance

Today I learned quite a lot, from how context windows work to the capabilities of newer models.

I also had a short conversation with a friend about what it would actually take to integrate multiple AI agents into one system.

The more I think about it, the more I realize that connecting several models together is not the hardest part.

The real difficulty lies in managing the information produced during collaboration.

Delegating Work Does Not Eliminate Context Cost

Suppose Claude acts as the primary model and delegates implementation tasks to Codex, which I will refer to as C, and Gemini, which I will refer to as G.

C or G may complete a coding task and return the result to Claude. However, Claude still has to read the returned code, understand what changed, evaluate the outcome, and decide what should happen next.

This means that although the implementation work has been delegated, the information-processing cost of the primary model has not disappeared.

In some cases, it may actually increase.

If every sub-agent returns large amounts of code, logs, explanations, and execution history, the primary model has to process all of that information again. The system may distribute the work successfully while still consuming more tokens overall.

So the central problem is not simply:

How do we connect multiple models?

It is:

How do we manage the information generated while those models collaborate?

Why Traditional RAG Is Not Enough

After discussing this with my friend, I began to feel that the missing layer is not just a simple RAG system.

Traditional retrieval-augmented generation is usually designed to retrieve relevant information from an external knowledge base.

The problem here is different.

A multi-agent system continuously produces its own information:

• task instructions
• implementation results
• code changes
• error messages
• decisions
• rejected approaches
• progress updates
• temporary observations

The system needs more than retrieval.

It needs a way to evaluate, filter, compress, store, and selectively recall this information.

Instead of inserting everything back into the context window, the system should first determine what is actually important.

High-value information should be preserved and assigned greater weight. Low-value information can be ignored, compressed, or stored only temporarily.

The primary model should not have to reread the entire codebase, every execution log, and the complete conversation history each time it makes a decision.

It should receive only the information that is relevant to the current task.

An Information Evaluation Layer

A few days ago, I saw someone share the idea of an “information evaluation system,” and I found it very useful.

The basic idea is to assign value to different types of information.

For example, the following information is usually more important than ordinary conversation:

• direct instructions from the user
• important changes in system state
• major task milestones
• comparisons between competing approaches
• errors that affect future implementation
• decisions that constrain later work

However, I do not think the system should preserve every intermediate step.

The most valuable information is usually the information that will influence future decisions.

For example:

• Which solution was ultimately selected?
• Why was another solution rejected?
• What caused the previous error?
• What is the current state of the task?
• What constraints must future agents respect?
• What remains unfinished?

These records are far more useful than preserving every intermediate message or every step of the reasoning process.

Memory Should Be Decision-Oriented

This suggests that agent memory should not simply be a complete archive.

It should be decision-oriented.

A useful memory layer might separate information into several categories:

Persistent information

Information that should remain available across the entire project:

• user requirements
• architectural decisions
• important constraints
• accepted conventions
• final implementation choices

Task-level information

Information that matters during the current stage of work:

• current objectives
• active files
• recent changes
• unresolved errors
• dependencies between tasks

Temporary information

Information that may be useful briefly but does not need to remain permanently:

• raw logs
• exploratory outputs
• unsuccessful intermediate attempts
• repetitive explanations

Discardable information

Information that has no meaningful effect on future decisions and does not need to be retained.

The challenge is determining when information should move from one category to another.

A temporary error may become persistent knowledge if it reveals an important architectural limitation. A rejected approach may be worth saving if another agent might otherwise repeat the same mistake.

Information Governance as the Coordination Layer

I am increasingly convinced that the hardest part of a multi-agent system is not the intelligence of the individual models.

It is the governance layer between them.

The system must continuously answer questions such as:

• What should be recorded?
• What should be summarized?
• What can be discarded?
• What should receive greater importance?
• What information should be recalled for the next task?
• Which agent needs access to which information?
• How much detail does the primary model actually need?

If these decisions are handled poorly, multi-model collaboration may become more expensive and more chaotic than using a single model.

Every agent may perform its assigned task correctly, while the overall system still fails because too much information is duplicated, repeated, and passed between models without structure.

In that sense, the coordination problem is not only about routing tasks.

It is about routing information.

The Real Bottleneck

Multi-agent systems are often described as a way to divide labor between specialized models.

That is true, but dividing labor also creates communication overhead.

The more agents a system contains, the more carefully it must manage what they communicate.

Without an effective information layer, every additional model may create more context, more duplication, and more token consumption.

With a strong information governance system, however, agents may be able to collaborate through concise state updates, structured decisions, and selectively retrieved memory.

The goal is not to make every model know everything.

The goal is to make sure that each model knows exactly what it needs, when it needs it.

That may be the real foundation of an efficient multi-agent system.