Asynchronous Session Compression with Checkpoint Segments

QoL Update: Asynchronous Session Compression with Checkpoint Segments

Problem

Currently, session compression is triggered together with the latest user message. This temporarily blocks the Composer and/or the active message stream.

This creates noticeable latency in the UX flow, even though compression does not technically need to run synchronously with the current user input.

Goal

Session compression should be decoupled from the active Composer and streaming path.

Instead of starting compression synchronously when the user sends a new message, the system should use checkpoint-based asynchronous compression in the background.

Proposed Behavior

The session should be split into fixed token windows, for example 256k tokens per compression window.

Once a window is complete, the system creates a compression checkpoint:

• session.compression.1
• session.compression.2
• session.compression.N

The last relevant message from the previous window should be copied or referenced into the next window to prevent a semantic flow break.

This keeps the transition between compressed history and the active session understandable.

While the user continues working, compression of the completed 256k-token window runs in the background.

The active Composer, message stream, and current session remain responsive.

Example

#BEGIN QOL_ASYNC_SESSION_COMPRESSION_SPEC
Active Session:
- Current work continues normally
- Composer remains usable
- Stream is not blocked

Checkpoint:
- 256k token window is reached
- Window is marked as a compression candidate
- Last relevant context message is carried into the next window

Background Job:
- session.compression.1 is generated
- session.compression.2.scrap can be prepared in parallel
- Compression runs independently from the active input/output flow

Result:
- Compressed context is atomically committed after successful completion
- Active session uses the raw context or the latest valid committed compression until then
- Composer and stream are never blocked by an unfinished compression job
#END QOL_ASYNC_SESSION_COMPRESSION_SPEC

Technical Requirements

1. Checkpoint-Based Segmentation

• Sessions are not compressed only at the end or during a new user message.
• Instead, a compression candidate is created once a fixed token boundary is reached.

1. Asynchronous Background Compression

• Compression runs as a background task.
• The Composer must not wait for it.
• Streaming must not be blocked.

1. Flow Preservation

• The last relevant message or a short transition context is carried into the next segment.
• This preserves semantic continuity between compression windows.

1. Atomic Commit

• A completed compression replaces raw context only after successful validation.
• Failed or incomplete compression outputs remain isolated as .scrap, .pending, or temporary state.

1. Scrap / Draft State

• Running compression jobs are stored as temporary artifacts, for example:

• session.compression.2.scrap
• session.compression.2.pending
• Only after validation should they become:

• session.compression.2

1. No UX Blocking

• The user can continue writing, sending, and receiving streamed output while compression runs.
• Composer and stream should only read from already committed compression states.

Result

Session compression changes from a synchronous blocking process into an asynchronous checkpoint system.

This keeps the UX responsive while large context windows are compressed in the background.

The system can still support large token windows without interrupting the active workflow every time the user sends a new message.