Skip to content

[BUG] Failed automatic summarization can clear existing chat history #4321

Description

@ssynb

Required prerequisites

  • I have read the documentation.
  • I searched Issues, PRs, and Discussions for summarization failure clears memory, summary empty memory, context compression data loss, and _update_memory_with_summary. I found related work on summary backends and callbacks, but no report covering this failure path.

What version of camel are you using?

master at 8c791b7b9cf7deab56cb5a92818c34499af9097f (the current preview release is 0.2.91a7).

System information

This is a provider- and OS-independent control-flow issue in ChatAgent. I verified it against the source at the commit above. No external model or optional toolkit is required to reach the destructive update once summarization returns an empty result.

Problem description

Automatic context compression rewrites memory even when summary generation fails.

summarize() and asummarize() catch model errors, empty responses, and empty summary content, then return a result whose summary is still the initial empty string. Both _get_context_with_summarization() variants subsequently call _update_memory_with_summary(summary.get("summary", "")) without checking status or the summary content.

_update_memory_with_summary("") clears the existing memory first, writes an empty assistant message, and restores only the last user message. A transient summarizer error can therefore discard the working conversation instead of leaving the pre-compression memory intact.

Relevant source paths:

  • Failure returns: camel/agents/chat_agent.py (summarize / asummarize)
  • Unconditional update: _get_context_with_summarization and _get_context_with_summarization_async
  • Destructive rewrite: _update_memory_with_summary

Reproducible example code

The smallest source-level reproduction does not require a live provider. Starting with any configured ChatAgent that already has more than one history message:

before, _ = agent.memory.get_context()

# This is the exact value passed by the automatic compression path when
# summarize()/asummarize() reports an exception or an empty model response.
failed_result = {
    "summary": "",
    "status": "Failed to generate summary using model: transient error",
}
agent._update_memory_with_summary(failed_result.get("summary", ""))

after, _ = agent.memory.get_context()

assert after == before
# Fails: the prior history was cleared; only an empty assistant message and
# the rewritten last user message remain (plus the restored system message).

The automatic callers currently perform the same last two steps after a failed summarize() / asummarize() result.

Expected behavior

Compression should be atomic with respect to memory: if summary generation fails or returns empty content, the original memory must remain unchanged. The caller should surface an explicit compression failure (or continue with the original context when safe), but it should not commit an empty summary.

Additional context

A minimal fix direction would be to validate both the summary result status and non-empty summary content before calling _update_memory_with_summary. A regression test should cover a summarizer exception and an empty msgs/empty-content response, asserting that every original memory record is preserved.

Related but non-duplicate work: #3374 (unified summarization backend), #3468 (last-user preservation), and #3976 (summary callback).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions