Automatic context compaction in Codex Desktop is extremely slow, sometimes taking around a full minute

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #23998
• #24332

Powered by Codex Action

Solution: Fix Extremely Slow Context Compaction in Codex Desktop

Issue: #24372 — Automatic context compaction in Codex Desktop is extremely slow, sometimes taking around a full minute.

Related Issues:

• #24332 — Allow compaction-specific reasoning effort and service tier (enhancement proposal with prepared implementation)
• #23998 — Codex app is much slower after latest update (duplicate)

---

1. Root Cause Analysis

1.1 Primary Root Cause: Compaction Inherits Full Reasoning Effort and Service Tier

The core issue is that context compaction reuses the same reasoning_effort and service_tier settings as normal conversation turns. This means:

1. High reasoning effort on compaction: When a user has model_reasoning_effort = "high" configured for normal turns, compaction also uses "high" reasoning effort. Compaction is a summarization task that does not benefit from deep reasoning — it should use minimal or no reasoning effort.

1. Same service tier for compaction: When a user has service_tier = "default" or "priority", compaction requests also use that tier. Compaction is a batch-like operation that does not need priority routing, and using the fast tier would significantly reduce latency.

1.2 Evidence from Source Code

compact.rs — Local/inline compaction (line ~544-546):

turn_context.reasoning_effort,          // ← Inherited from session config
turn_context.reasoning_summary,
turn_context.config.service_tier.clone(), // ← Inherited from session config

compact_remote.rs — Remote compaction v1 (line ~195-201):

CompactConversationRequestSettings {
    effort: turn_context.reasoning_effort,  // ← Inherited from session config
    summary: turn_context.reasoning_summary,
    service_tier: if sess.services.auth_manager.auth_mode() == Some(AuthMode::ApiKey) {
        None
    } else {
        turn_context.config.service_tier.clone()  // ← Inherited from session config
    },
},

compact_remote_v2.rs — Remote compaction v2 (line ~298-300):

turn_context.reasoning_effort,          // ← Inherited from session config
turn_context.reasoning_summary,
turn_context.config.service_tier.clone(), // ← Inherited from session config

1.3 Why This Causes ~1 Minute Delays

• High reasoning effort triggers extended thinking chains, which can add 30-60+ seconds to any request.
• Default/priority service tier may route to heavily loaded capacity, whereas the fast tier uses dedicated lower-latency infrastructure.
• Compaction is called multiple times during long-running sessions, so each ~60s compaction adds up to significant total delay.
• The compaction task blocks the entire session — no coding work can proceed while compaction is in progress.

1.4 Secondary Contributing Factors

1. No streaming progress feedback: During compaction, the UI shows "Context automatically compacted" with no progress indicator, making the delay feel longer.

1. Retry overhead: The retry logic in compact_remote_v2.rs (up to 2 retries with exponential backoff) can add additional delay when requests fail.

1. Large context window: When compaction is triggered, the entire conversation history (potentially 100K+ tokens) must be sent to the model, increasing processing time.

---

2. Step-by-Step Fix Approach

Phase 1: Add Compaction-Specific Configuration Fields (Primary Fix)

Add two new optional configuration fields to ConfigToml that allow users to override reasoning effort and service tier specifically for compaction:

• compact_model_reasoning_effort — Reasoning effort for compaction (defaults to "low")
• compact_service_tier — Service tier for compaction (defaults to "fast")

Phase 2: Wire Configuration Through to Compaction Code

Modify the three compaction implementations to use the new compact-specific settings when available.

Phase 3: Sensible Defaults

Default compaction to use reasoning_effort = "low" and service_tier = "fast" when the user has not explicitly configured compact-specific settings. This provides immediate improvement for all users.

---

3. Code Changes

3.1 Add Configuration Fields (codex-rs/config/src/config_toml.rs)

// Add to ConfigToml struct alongside existing model_reasoning_effort and service_tier fields:

/// Reasoning effort to use for context compaction.
/// When unset, compaction defaults to "low" reasoning effort regardless of
/// the session-level model_reasoning_effort setting.
pub compact_model_reasoning_effort: Option<ReasoningEffort>,

/// Service tier to use for context compaction.
/// When unset, compaction defaults to "fast" service tier regardless of
/// the session-level service_tier setting.
pub compact_service_tier: Option<String>,

3.2 Add Config Resolution Methods (codex-rs/core/src/config/mod.rs)

impl Config {
    /// Returns the reasoning effort to use for compaction.
    /// Defaults to "low" if not explicitly configured.
    pub fn compact_reasoning_effort(&self) -> Option<ReasoningEffortConfig> {
        // If compact-specific effort is set, use it
        if let Some(effort) = self.compact_model_reasoning_effort {
            return Some(effort.into());
        }
        // Default compaction to low reasoning effort
        Some(ReasoningEffortConfig::Low)
    }

    /// Returns the service tier to use for compaction.
    /// Defaults to "fast" if not explicitly configured.
    pub fn compact_service_tier(&self) -> Option<String> {
        // If compact-specific tier is set, use it
        if let Some(tier) = &self.compact_service_tier {
            return Some(normalize_service_tier(tier));
        }
        // Default compaction to fast tier
        Some("fast".to_string())
    }
}

3.3 Update Local Compaction (codex-rs/core/src/compact.rs)

In drain_to_completed, replace the inherited settings with compact-specific ones:

async fn drain_to_completed(
    sess: &Session,
    turn_context: &TurnContext,
    client_session: &mut ModelClientSession,
    turn_metadata_header: Option<&str>,
    prompt: &Prompt,
) -> CodexResult<()> {
    let mut stream = client_session
        .stream(
            prompt,
            &turn_context.model_info,
            &turn_context.session_telemetry,
            turn_context.config.compact_reasoning_effort(),  // ← Use compact-specific effort
            turn_context.reasoning_summary,
            turn_context.config.compact_service_tier(),      // ← Use compact-specific tier
            turn_metadata_header,
            &InferenceTraceContext::disabled(),
        )
        .await?;
    // ... rest unchanged
}

3.4 Update Remote Compaction v1 (codex-rs/core/src/compact_remote.rs)

let mut new_history = sess
    .services
    .model_client
    .compact_conversation_history(
        &prompt,
        &turn_context.model_info,
        CompactConversationRequestSettings {
            effort: turn_context.config.compact_reasoning_effort(),  // ← Use compact-specific
            summary: turn_context.reasoning_summary,
            service_tier: if sess.services.auth_manager.auth_mode() == Some(AuthMode::ApiKey) {
                None
            } else {
                turn_context.config.compact_service_tier()  // ← Use compact-specific
            },
        },
        &turn_context.session_telemetry,
        &compaction_trace,
    )

3.5 Update Remote Compaction v2 (codex-rs/core/src/compact_remote_v2.rs)

In run_remote_compaction_request_v2:

let result = match client_session
    .stream(
        prompt,
        &turn_context.model_info,
        &turn_context.session_telemetry,
        turn_context.config.compact_reasoning_effort(),  // ← Use compact-specific
        turn_context.reasoning_summary,
        turn_context.config.compact_service_tier(),      // ← Use compact-specific
        turn_metadata_header,
        &InferenceTraceContext::disabled(),
    )
    .await

3.6 Update Config Schema (codex-rs/core/config.schema.json)

Run just write-config-schema after adding the new fields to regenerate the JSON schema.

---

4. Testing Strategy

4.1 Unit Tests

Test compact-specific config resolution:

#[test]
fn compact_reasoning_effort_defaults_to_low() {
    let config = Config::builder().build().unwrap();
    assert_eq!(config.compact_reasoning_effort(), Some(ReasoningEffortConfig::Low));
}

#[test]
fn compact_service_tier_defaults_to_fast() {
    let config = Config::builder().build().unwrap();
    assert_eq!(config.compact_service_tier(), Some("fast".to_string()));
}

#[test]
fn compact_reasoning_effort_uses_explicit_config() {
    let config = Config::builder()
        .compact_model_reasoning_effort(Some(ReasoningEffort::High))
        .build()
        .unwrap();
    assert_eq!(config.compact_reasoning_effort(), Some(ReasoningEffortConfig::High));
}

#[test]
fn compact_service_tier_uses_explicit_config() {
    let config = Config::builder()
        .compact_service_tier(Some("priority".to_string()))
        .build()
        .unwrap();
    assert_eq!(config.compact_service_tier(), Some("priority".to_string()));
}

Test TOML deserialization:

#[test]
fn config_toml_compact_reasoning_effort_deserializes() {
    let config: ConfigToml = toml::from_str(
        r#"compact_model_reasoning_effort = "high""#
    ).expect("compact_model_reasoning_effort should deserialize");
    assert_eq!(config.compact_model_reasoning_effort, Some(ReasoningEffort::High));
}

#[test]
fn config_toml_compact_service_tier_deserializes() {
    let config: ConfigToml = toml::from_str(
        r#"compact_service_tier = "fast""#
    ).expect("compact_service_tier should deserialize");
    assert_eq!(config.compact_service_tier, Some("fast".to_string()));
}

4.2 Integration Tests

Test remote compaction uses compact-specific settings:

#[tokio::test]
async fn remote_manual_compact_uses_compact_specific_service_tier_and_reasoning_effort() {
    // Verify that when compact-specific settings are configured,
    // the remote compaction request uses those settings instead of
    // the session-level settings.
}

4.3 Manual Testing

1. Baseline measurement: Run a long session with model_reasoning_effort = "high" and measure compaction latency.
2. After fix: Verify compaction completes in < 10 seconds with default compact settings.
3. Override test: Set compact_model_reasoning_effort = "high" and verify compaction uses high effort (backward compatibility).

4.4 Regression Tests

• Ensure existing sessions without compact-specific config still work (backward compatibility).
• Verify that API key auth mode still skips service tier for compaction (existing behavior).
• Verify all three compaction paths (local, remote v1, remote v2) use the new settings.

---

5. Impact Assessment

5.1 Performance Impact

| Scenario | Before Fix | After Fix | Improvement |
|----------|-----------|-----------|-------------|
| High reasoning effort, default tier | ~60s per compaction | ~5-10s per compaction | 6-12x faster |
| Standard reasoning effort, default tier | ~15-30s per compaction | ~5-10s per compaction | 2-3x faster |
| User explicitly sets high compaction effort | ~60s per compaction | ~60s per compaction | No change (opt-in) |

5.2 Cost Impact

• Reduced cost: Using service_tier = "fast" and reasoning_effort = "low" for compaction reduces token consumption and compute cost per compaction event.
• No impact on normal turns: Regular conversation turns continue to use user-configured settings.

5.3 Backward Compatibility

• Fully backward compatible: The new config fields are optional. When unset, compaction defaults to low reasoning effort and fast service tier — a sensible improvement over the current behavior.
• Users can opt out: Users who prefer the old behavior can set compact_model_reasoning_effort and compact_service_tier to match their session-level settings.

5.4 Risk Assessment

| Risk | Likelihood | Mitigation |
|------|-----------|------------|
| Compaction quality degrades with low reasoning | Low | Compaction is a summarization task; low reasoning is sufficient. Users can override. |
| Fast tier capacity limits | Low | Fast tier is designed for low-latency batch-like operations. Fallback to default tier if needed. |
| Config parsing errors | Very Low | New fields are optional with sensible defaults. |
| Breaking existing integrations | Very Low | Changes are additive only; no existing behavior is removed. |

5.5 Files Modified

| File | Change Type | Lines Affected |
|------|------------|----------------|
| codex-rs/config/src/config_toml.rs | Add fields | +10 |
| codex-rs/core/src/config/mod.rs | Add resolution methods | +20 |
| codex-rs/core/src/compact.rs | Use compact settings | ~3 |
| codex-rs/core/src/compact_remote.rs | Use compact settings | ~3 |
| codex-rs/core/src/compact_remote_v2.rs | Use compact settings | ~3 |
| codex-rs/core/config.schema.json | Auto-regenerated | +20 |
| codex-rs/core/src/config/config_tests.rs | Add tests | +50 |
| codex-rs/core/tests/suite/compact_remote.rs | Add integration tests | +30 |

Total estimated impact: ~140 lines across 8 files.

---

6. Alternative Approaches Considered

6.1 Hardcode Low Effort + Fast Tier for All Compaction

Pros: Simplest implementation, immediate benefit for all users.

Cons: No user override capability; users who need high-quality compaction cannot opt in.

Verdict: Not recommended. The config-based approach (Phase 1) is better because it provides both automatic improvement and user control.

6.2 Auto-Detect Compaction Complexity

Pros: Dynamically adjusts reasoning effort based on context size.

Cons: Complex to implement correctly; heuristic-based; may produce inconsistent results.

Verdict: Over-engineering. The config-based approach is simpler and more predictable.

6.3 Streaming Progress Indicator for Compaction

Pros: Improves perceived latency.

Cons: Does not address the root cause (actual latency); only masks the problem.

Verdict: Complementary improvement, not a replacement for the primary fix.

---

7. Recommended Implementation Order

1. Step 1: Add compact_model_reasoning_effort and compact_service_tier to ConfigToml.
2. Step 2: Add resolution methods to Config with smart defaults (low effort, fast tier).
3. Step 3: Update compact.rs (local compaction) to use compact-specific settings.
4. Step 4: Update compact_remote.rs (remote compaction v1) to use compact-specific settings.
5. Step 5: Update compact_remote_v2.rs (remote compaction v2) to use compact-specific settings.
6. Step 6: Regenerate config schema with just write-config-schema.
7. Step 7: Add unit and integration tests.
8. Step 8: Run just fix -p codex-core and just test -p codex-core.

---

8. References

• Original issue: https://github.com/openai/codex/issues/24372
• Enhancement proposal with prepared implementation: https://github.com/openai/codex/issues/24332
• Prepared branch: https://github.com/Cccceb/codex/tree/compact-specific-reasoning-service-tier
• Commit: https://github.com/Cccceb/codex/commit/64b6b1e00ad40558c9e4d78c4a6d17b4d01f3b7a
• Duplicate issue: https://github.com/openai/codex/issues/23998

Thanks for reporting this problem. Until recently, Codex used a separate endpoint and server-side logic for compaction. We recently switched to a more robust approach where the compaction logic is moved locally using the same endpoint as normal turns. This eliminates "remote compaction" errors and increases the stability and reliability of compaction operations. If you see further problems with compaction, please use /feedback and open a new bug report.