The chat window is not the service system. It is one channel into a system that still needs identity, knowledge, business rules, case ownership, human judgment, recovery paths, and operating evidence. A fluent answer can make the experience feel complete while the actual work—opening the right case, preserving consent, or getting a person to accept an escalation—quietly fails behind it.
Design from the case lifecycle outward. Chat is the conversational layer through which a customer, patient, employee, or agent enters it.
The architecture behind the conversation
| Layer | Responsibility | Primary failure to control |
|---|---|---|
| Channel and identity | Establish where the message came from and who is speaking | Treating an unverified claim as an authenticated identity |
| Conversation state | Preserve turns, intent, tool results, and resumable context | Losing context across sessions, devices, or handoff |
| Grounding and rules | Retrieve approved facts and apply deterministic policy | A plausible answer replacing a governed decision |
| Case and workflow | Create durable work with ownership, priority, and evidence | A conversation ending without accountable follow-through |
| Human service | Route risk with enough context for a person to continue | A transfer that makes the user repeat the story |
1. Start with channel and identity
Web, authenticated portal, mobile, SMS, and messaging channels have different identity strength and continuity. Decide what an anonymous visitor may do, what requires verification, and when the conversation must step up authentication. A name, email address, account number, or date of birth typed into chat is a claim—not proof by itself.
Bind verified sessions to the correct Salesforce person and account without exposing more record data than the user needs. Define token expiry, logout, device changes, and what happens when identity cannot be resolved. Salesforce documents a verified-user pattern for Enhanced Web Chat using identity tokens; the implementation still needs your own authentication and authorization decisions.
2. Separate conversation state from customer and case state
A transcript is not a customer record, and a messaging session is not automatically a case. Keep a correlation ID across the channel, orchestration layer, retrieval calls, tool actions, and Salesforce records. Store the minimum conversation state needed to resume safely: verified identity status, expressed intent, facts already collected, disclosures shown, consent captured, actions attempted, and current owner.
Define session boundaries. If someone returns tomorrow, decide whether to reopen the prior service request, start a new one, or ask them to confirm. Do not rely on the model's context window as the system of record. Durable state belongs in governed application data.
3. Ground answers in an approved knowledge path
Retrieval should start with a source inventory: Salesforce records, Knowledge, policy documents, product documentation, case history, and any external repository. For each source, define an owner, access model, freshness expectation, and whether its content may be shown to this audience. Retrieval must respect identity and purpose; finding a relevant passage does not mean the current user is allowed to see it.
Return source identifiers and versions with retrieved passages. Ask the model to answer from that evidence, cite it where useful, and say when the evidence is missing or conflicting. Evaluate retrieval and response generation separately: the system cannot produce a grounded answer if the right source never entered the context.
4. Keep business policy deterministic
Use the model for language understanding, summarization, and drafting. Use explicit rules or governed tools for eligibility, service level, identity requirements, jurisdiction, consent, priority, entitlement, and actions that change a record. The model can propose an intent or field value; a deterministic validation layer should decide whether it is allowed.
Give every tool a narrow contract: required inputs, allowed values, authorization context, validation, idempotency key, response, and known errors. Separate read tools from write tools. The user should see a confirmation before consequential changes, and the audit trail should show what the model requested, what the rule layer allowed, and what Salesforce actually committed.
5. Create a case when durable work begins
Decide which intents warrant a case, which should update an existing case, and which can be resolved without one. Create or update by a stable request key so retries do not produce duplicates. A useful case record carries the verified person and account where available, channel, intent, category, priority inputs, concise summary, transcript reference, consent state, actions already attempted, and the reason for escalation.
Case creation must return a real result. If the transaction fails, tell the user the request was not recorded, retain enough state to retry safely, and route the exception. Never show “your case is open” merely because the model generated the sentence.
6. Draw consent and sensitive-data boundaries before prompting
Minimize what the chat collects, retrieve only what the task requires, and define which fields or document types must never enter model context. Review model-provider terms, retention settings, data residency, access controls, encryption, logging, redaction or masking, and deletion behavior for the exact architecture—not for “AI” in the abstract.
For healthcare, financial, employment, or other sensitive workflows, place clear limits on advice and automated action. Consent language, emergency instructions, record-retention duties, and human-review requirements depend on jurisdiction, use case, contracts, and policy. This is general architecture guidance, not legal, medical, privacy, or compliance advice. Have the appropriate professionals approve the operating boundary.
7. Design human handoff as a continuation
Trigger handoff when the user asks, identity cannot be verified, the system lacks evidence, confidence is low, a sensitive or urgent intent appears, a tool repeatedly fails, or policy requires a person. Route by skill, priority, language, entitlement, and availability, with a queue and timeout fallback when no one can accept.
The nurse or support agent should receive a compact workspace: the customer and case, identity status, transcript and summary, collected fields, cited sources, tool outcomes, risk flags, and a suggested next action. The summary helps navigation; it does not replace the source conversation. Clinical, policy, exception, and other accountable decisions remain with the qualified person.
8. Observe and evaluate the complete service loop
Log each retrieval, rule decision, tool call, case mutation, escalation, and user-visible outcome under the correlation ID. Protect the logs with the same care as the underlying data. Monitor task completion, retrieval relevance, grounded-answer quality, case-creation success, duplicate rate, handoff accuracy, time to human acceptance, repeat contact, and recovery from failure. “Containment” alone can reward a bot that prevents users from reaching help.
Build an evaluation set from approved scenarios: ordinary questions, missing and conflicting knowledge, unauthorized requests, ambiguous identity, prompt injection, sensitive content, urgent escalation, tool timeout, partial write, unavailable agent, and resumed conversation. Assert both the response and the system state afterward.
Failure and recovery are part of the experience
Define a user-safe response for every dependency: retrieval unavailable, model timeout, invalid tool input, Salesforce write failure, duplicate request, and routing outage. Retry only operations that are safe to repeat. Use idempotency for writes, circuit breakers for unhealthy dependencies, and a visible exception queue for unresolved work. If automation cannot complete the task, preserve the context and offer a real next step rather than improvising success.
A phased launch that earns more scope
- Foundation: map intents, identity, sources, data boundaries, case states, handoff policy, owners, and evaluation cases.
- Grounded guidance: launch one audience and a narrow, read-only knowledge domain with human escalation always available.
- Governed case creation: add validated intake and idempotent case creation, then prove routing, acceptance, and recovery.
- Selective actions: add one write action at a time, with confirmation, evaluation, monitoring, and rollback before the next action is exposed.
This architecture belongs inside a broader service operating system and case-intake design. For the AI, Salesforce, and governance layers together, see our Salesforce AI consulting service. Or bring us the service journey you need to make conversational.
Official sources
- Salesforce Developers: User Verification in Enhanced Web Chat
- Salesforce Help: Einstein Trust Layer: Designed for Trust
- Salesforce Help: Advanced Routing with Omni-Channel Flows