Reliable context for business AI agents
Design reliable context for business AI agents with identity, authoritative sources, permissions, task state, tool controls, evidence, and evaluation.
A business AI agent can read a current customer record without copying the entire CRM into a vector database. That does not settle whether it may send the returned fields to an external model, retain them in a conversation, or use them to change the account.
Those are separate decisions. Reliable context starts with an authenticated user, a defined task and an approved route from source data to an answer or action. The connector is one part of that route.
This guide explains how to choose the retrieval boundary and test what happens around it. Our AI automation services cover the design and implementation of bounded workflows; a specific deployment still needs its own access, processing and acceptance checks.
Decide what “data stays in place” means
Start by asking the data owner what is prohibited. It may be a bulk replica, storage outside an approved region, disclosure to another processor, or any result leaving a particular environment. These restrictions lead to different architectures.
Live retrieval avoids one kind of copy: a separate searchable collection of source content. It still returns something. Selected fields, excerpts and even aggregates may contain protected information. Trace that result through the agent host, model service, conversation history, caches, tracing and support logs before calling the design no-copy.
The distinction exists in current products. Microsoft’s connector overview separates synced connectors, which index content into Microsoft Graph, from federated connectors, which fetch live data. Its federated connector documentation describes user-scoped, read-only access without indexing the external content into Microsoft 365. Check the specific connector and supported experience; this is not a promise that every source has a suitable connector or that returned results are never processed or retained.
| Route | When it can fit | Boundary to verify |
|---|---|---|
| Index approved content | Stable documents need search across many sources. | Which content and access metadata are copied, where they are stored, and how changes or removals propagate. |
| Query a live API or approved data view | The answer depends on current records or a bulk replica is unsuitable. | Allowed fields, user scope, result size, processing destinations and behavior during an outage. |
| Process inside the approved environment | Policy prohibits sending the required result to the proposed external service. | The actual model, agent host, dependencies and logs all satisfy that boundary. An on-premises source alone is insufficient. |
These are design options, not certifications. If the required result cannot enter the proposed model environment, stop that route. Use an approved report or redesign the processing location with the responsible security and data owners. A connector cannot override that decision.
Begin with identity and authority
Resolve the caller’s identity from authentication, not a name or department typed into the conversation. Establish the organization, source access and permitted operations before retrieving sensitive records. Read access does not imply permission to export, approve or update them.
If a connector uses a service account, document how its wider source access is narrowed to the caller. Test with two roles that should see different results and with an account whose access has been revoked. A successful administrator demo cannot establish ordinary-user isolation.
Authorization belongs in application and source-system controls. The model may interpret intent, but it should not decide whether its caller has permission. For MCP connections, the security guidance also requires tokens issued for the MCP server and forbids accepting and passing through tokens intended for other resources. An authenticated connection is not permission to reuse a credential for any downstream API.
Give the read operation a useful contract
A tool called “search everything” leaves too many business decisions implicit. Prefer an operation whose scope can be checked: the current status of a permitted account, approved policy passages for a jurisdiction, or open work assigned to the caller’s team.
For an illustrative account-status lookup, define:
- Input: an account identifier the caller may access, with validated filters and a result limit.
- Output: the relevant status fields, source record identifier and retrieval time. Include the source’s modification or effective date when available.
- Exclusions: credentials, unrelated accounts and personal fields not needed for this decision.
- Failure: distinguish no matching record, denied access, unavailable source and stale fallback. Do not turn every failure into an empty result.
This is a proposed contract, not a tested customer integration. A warehouse or mainframe connection needs the same explicit boundary; do not substitute unrestricted database access for a missing business operation. Begin read-only when the task is to answer a question.
Distinguish knowledge from live state
A policy, a meeting commitment and a current account balance have different authority and freshness requirements. Retrieve each from the source responsible for it. A summary from last week should not override today’s system-of-record field merely because its prose is easier to match.
Decide in advance whether a stale result is acceptable for the task. If it is, expose its age and limit what the agent may do with it. If it is not, an unavailable source should block the dependent answer or action. “Live” retrieval also has limits: records can change between a read and a later write.
The AI knowledge-management guide covers source ownership, competing versions and the correction process. Those decisions should be available to the retrieval system, not rediscovered from every conversation.
Separate an answer from permission to act
A read connector may be sufficient for a status brief. Updating a record introduces another permission check, a valid state transition and a recovery obligation. Do not add write tools simply because the platform supports them.
For a consequential change, show the target record, proposed fields and relevant evidence before approval. Recheck authorization and current record state when the approved action runs. If the record has changed, require a fresh decision rather than applying an obsolete preview.
Keep the action record outside the conversation: initiating request, approval, attempted operation, returned identifier and unresolved work. Use native idempotency support where the destination provides it. Otherwise define reconciliation or duplicate checks appropriate to that operation. A timeout can leave the outcome unknown; blindly repeating a create request is not recovery.
Treat retrieved instructions as untrusted content
OWASP’s prompt-injection guidance recommends separating external content, limiting privileges and reviewing high-risk actions. It also warns that defenses are not foolproof. Labeling a retrieved document “evidence” helps define its role, but the enforcement must happen outside the model too.
A document that asks the assistant to send records elsewhere must not enable a new destination or tool. Keep credentials in the adapter, validate arguments and restrict what each operation can read or change. Include malicious source text in the test set, not only direct hostile user prompts.
Inspect the full path before increasing autonomy
An acceptable final answer can conceal an unauthorized intermediate read. Test the retrieval, processing and action boundaries separately, with evidence for each:
- An authorized user gets the expected fields and source reference, while a different role cannot retrieve the restricted record.
- Revoking access blocks a new retrieval, including cached-result paths. Explain what happens to information already disclosed in a previous answer.
- An unavailable source produces the agreed failure or visibly dated fallback, not an invented current status.
- A hostile document cannot expand tool access, export data or change the user’s requested task.
- A record changed after approval is checked again; an unknown write outcome stays unresolved until reconciliation establishes what happened.
These are acceptance cases to adapt, not a report of completed tests. Record the source and connector versions, expected outcome, observed result and failure owner. Repeat relevant cases when permissions, schemas, model behavior or business rules change. NIST’s AI RMF Core treats risk management as continuous; its functions are not a one-time launch checklist.
Begin with a question the business needs answered under those controls. If read-only retrieval solves it, keep that boundary. Add an action only when its owner can explain the authority, expected result and recovery path. The Business Brain implementation offer is a place to scope that work, not evidence that an untested source or regulated deployment is already supported.
Questions people ask
What context does a business AI agent need?
It needs the authenticated user and permissions, the current task and entity state, authoritative evidence, available tools and schemas, business policy, action history, and a clear completion or escalation rule.
Is a large prompt enough for reliable agent context?
No. A prompt cannot replace live identity, permission checks, retrieval, structured state, tool authorization, provenance, and evaluation. Context is an application system around the model.
How should an AI agent handle conflicting sources?
It should prefer defined source authority, expose the conflict with dates and evidence, avoid unsupported action, and route the issue to the responsible owner when it cannot be resolved safely.
Can an agent use business data without copying it into a vector database?
Yes, a permitted live API or federated query can return selected results without building a separate vector index. The returned data still needs an approved processing destination, retention policy and access checks. No separate index does not mean no data leaves the source.