The AI customer service agent just gave an order number and lost it next turn—what usually causes this in 2026?

Conclusion up front: When an AI customer service agent loses the order number by the third turn, in most 2026 projects the model has not become less capable; the session state is not holding it: the session ID is unstable, only the current turn is passed, the summary strips out numbers, or the slots are not persisted. Check in the order “session ID—window—summary—slots” first, then decide whether to add a vector database; going straight to RAG often raises cost while still giving inaccurate answers.

Why agents still lose context in 2026

An LLM itself is stateless; every request is a new conversation to it. The dialogue system must pass historical messages, user identity, and key entities across requests by itself. In 2026, common channels include web pages, mini programs, apps, and H5, and each channel makes requests differently: a mini program page jump may reinitialize, an app may have multiple logged-in clients, and an H5 page may be reclaimed by the browser. As long as the session ID does not remain stable across requests, the backend cannot retrieve history, and the agent behaves as if it is meeting the user for the first time.

• The model is stateless: context is assembled by the application layer, not remembered by the model itself.
• Channel differences: mini programs, apps, and H5 have different session persistence capabilities and need unified encapsulation.
• Cost constraints: history cannot be inserted indefinitely; you must trade off between window and summary.

First distinguish: session memory and long-term knowledge are not the same thing

When the user says “change the address for that order from earlier,” that is session memory; when the user asks “how many days is the return policy,” that is long-term knowledge. Mixing the two leads to searching the current conversation in a vector database, or treating an old document as the current answer. To tell them apart, check whether the user is referring to information that appeared in the current turn or recent conversation.

Session memory belongs in a session table, cache, or context, while only long-term knowledge is suitable for a vector database. Once separated, troubleshooting can quickly determine whether it is an orchestration problem or a retrieval problem, instead of changing the model or adding a database right away.

• Session memory: recent messages, the order number just mentioned, current intent, temporary preferences.
• Long-term knowledge: product manuals, FAQ, policy terms, knowledge shared across users.
• Common misjudgment: the user says “it” or “the one above,” but the system searches a vector database and retrieves a pile of irrelevant documents.

Troubleshooting order: session ID, window, summary, and slots

In project delivery, session memory can be split into three layers: the immediate window, the session summary, and fact slots. The original text of recent turns is most accurate but consumes tokens; the summary saves tokens but loses details; slots are the most stable but cover only structured information. Check whichever layer has the problem.

1. Immediate window: keep the original text of the last 5 to 10 turns, truncate dynamically by token budget, and do not truncate only by turn count.
2. Session summary: once the window is exceeded, do rolling summaries; the summary must retain numbers, order numbers, negations, and time, and the template should be as fixed as possible.
3. Fact slots: extract user identity, order number, address, and preferences into structured fields and persist them; when the user changes their statement, the new value prevails.
4. Long-term knowledge: only general knowledge that spans sessions and users goes into a vector database, with version filtering added.

Suggested troubleshooting order: first check whether the session ID is stable, then whether the window passes only the current turn, then whether the summary and slots have lost key entities. This order keeps changes within a relatively small scope.

• Check the session ID: does the front end create a new ID on every request? Does the backend use the same ID to fetch history?
• Check the window: is only the current sentence passed? Is the window based on tokens rather than turn count?
• Check the summary and slots: does the summary lose numbers and negations? Are slots persisted and updatable?

Adding a vector database vs. changing conversation orchestration: how much do cost and timeline differ?

These two are often conflated. Changing conversation orchestration fixes the memory chain, while adding a vector database supplements knowledge retrieval capability; they solve different problems. In 2026 project delivery, if users only complain that the agent forgets what was just said, doing conversation orchestration first usually shows results faster; if the question is about policy details that exist only in documents, then the vector database needs to be scheduled.

• Changing conversation orchestration: typical timeline range is about 3 to 10 business days; the main cost is development labor, with limited increase in cloud cost; suitable for reference failures, multi-turn breakdowns, and slot loss.
• Adding a vector database: typical timeline range is about 2 to 6 weeks; it requires embedding models, a vector database, chunking, and retrieval tuning; typical monthly cloud cost range is about a few hundred to a few thousand RMB; suitable for scenarios with a large document volume and a high proportion of knowledge Q&A.
• Combining both: suitable for customer service and shopping assistant agents that need both multi-turn memory and knowledge retrieval; the suggested order is orchestration first, retrieval second, to avoid the retrieval layer masking memory layer problems.

A simple test can be used as a criterion: if the user says “that order from earlier” and the system cannot answer, check session memory first; if the user asks “how many days for returns” and the answer is wrong, check the knowledge base version and retrieval first. Only by counting the two types of problems separately can you know where the budget is going.

When it applies and when it does not

Not all AI agents need a complete memory system. For single-turn Q&A, translation, OCR recognition, and batch summarization, each request is independent, and forcing in a session table only adds complexity. Low-frequency internal tools are fine with a simple window plus session ID and do not need a vector database.

• Suitable for session memory: customer service, after-sales, shopping assistance, educational Q&A, ticket booking and changes, and other scenarios that need multi-turn references and state continuity.
• Suitable for adding a vector database: scenarios with many product documents, frequent policy updates, a high proportion of knowledge Q&A, and answers shared across users.
• No need for a memory database: single-turn tools, translation, image recognition, fixed FAQ, one-off tasks, and low-frequency internal small tools.
• Boundary reminder: if the user's question contains no reference and no state continuity, adding a memory layer only raises token cost and does not improve answer quality.

On the delivery floor: under a two-week constraint, which layer should be fixed first?

A common situation in projects: limited budget, timeline compressed to two weeks, and only one FAQ as material, while users often get stuck on “that order from earlier—the agent answers something irrelevant.” The approach is to first unify the session ID, add a sliding window and fact slots, and not rush to buy a vector database. The cost is that the front end must change request encapsulation and the backend must add a session table, with a typical rework range of about two to three days; but this avoids two to three weeks of vector database tuning and a monthly cloud cost, and after launch, reference-related problems drop noticeably.

Acceptance should not only look at whether the chat “sounds human”; it should use reproducible tests. In a reference test, have the user say “it” or “that one from earlier” and see whether it points to the correct object; in a slot breakpoint test, insert an unrelated topic in the middle, then return to ask about earlier information and see whether the slot has been overwritten. These tests can expose most memory problems before launch.

• Creating a new session ID on every request: all history is lost, and users feel the agent has amnesia.
• Passing only the current turn: the model cannot see prior text, and references easily fail.
• Overly aggressive summarization: numbers, order numbers, and negations are stripped out, so answers are naturally wrong.
• No version filtering in the vector database: old documents are recalled, and the agent answers with an already retired policy.
• Mindlessly stuffing in history: token cost rises, responses slow down, and irrelevant content may interfere.

Acceptance criteria can be set as: within five turns, key entity retention reaches a typical range of around 90%, reference tests pass, and single-turn token consumption stays within the budget range. If this cannot be achieved, fix orchestration first rather than continuing to tune prompts.

Common questions

Are an AI agent's memory and a vector database the same thing?

No. Memory manages what happened in the current session, while a vector database manages long-term knowledge shared across users. Mixing them leads to troubleshooting reference problems in the wrong direction and lets old documents pollute the current conversation.

How many turns of context are appropriate to keep in a multi-turn conversation?

A typical approach is to keep the original text of the last 5 to 10 turns, then layer on summaries and slots; adjust the exact number of turns according to the token budget—keep fewer when there are many long messages and more when there are many short messages.

What happens if the session ID is unstable?

Every request is treated by the backend as a new user, history cannot be retrieved, and the agent behaves as if it has amnesia. First check whether the front-end request reuses the same session ID, then check whether the backend retrieves history by it.

The agent cannot remember user preferences—is storing all chat logs enough?

Not necessarily cost-effective. Extract preferences into structured slots, such as language, size, and frequently used address, persist them, and inject them as needed; this usually saves more tokens and is more stable than inserting the full chat log every time.

After adding a vector database, the agent answers with a different product—what is going on?

A common cause is that the knowledge base lacks version filtering and product line isolation, so old or irrelevant documents are retrieved. First tag documents with version, product line, and effective time, then limit the retrieval scope.

If the agent is for customer service, shopping assistance, or tool-type dialogue, first troubleshoot in the order “session ID—window—summary—slots,” then evaluate the vector database; single-turn Q&A, translation, or low-frequency internal tools do not need a memory database. Before launch, use reference tests and slot breakpoint tests for acceptance, and check the context length and billing rules of the model you use against official documentation, so you do not spend money by mixing memory problems with knowledge problems.