Add built‑in exponential backoff & retry on OpenAI API rate‐limit errors

Resolved 💬 7 comments Opened Apr 17, 2025 by OneGitHubUsername Closed Apr 22, 2025
💡 Likely answer: A maintainer (tibo-openai, collaborator) responded on this thread — see the highlighted reply below.

Summary

Codex wrote this issue for me:
Currently any 429 "rate limit" response from the OpenAI API bubbles all the way out and causes the Codex‑CLI process to crash. We should instead catch rate‑limit errors, wait (with exponential back‑off), and retry automatically, so brief spikes don't kill the user's session.

Problem

  • Hitting the OpenAI API rate limit immediately aborts the CLI.
  • Users lose work/context when a single 429 becomes an unhandled exception.
  • There is no way to tune retry parameters or gracefully recover.

Proposal

  1. Introduce a helper, e.g. codex/utils/openai_backoff.py:

```python
import time
from openai.error import RateLimitError, OpenAIError

def openai_with_backoff(fn, args, max_retries=10, initial_delay=1.0, *kwargs):
"""
Wrap any OpenAI call (fn) so on RateLimitError it waits and retries
with exponential backoff (capped at e.g. 60s).
"""
delay = initial_delay
for attempt in range(max_retries):
try:
return fn(args, *kwargs)
except RateLimitError:
print(f"⏳ Rate limit hit, sleeping {delay:.1f}s (attempt {attempt+1}/{max_retries})")
time.sleep(delay)
delay = min(delay * 2, 60.0)
except OpenAIError:
raise
raise RuntimeError(f"OpenAI call failed after {max_retries} retries")
```

  1. Monkey‑patch the two main entrypoints in startup (e.g. in codex/cli.py):

```python
import openai
from codex.utils.openai_backoff import openai_with_backoff

_orig_chat = openai.ChatCompletion.create
openai.ChatCompletion.create = lambda args, *kw: openai_with_backoff(_orig_chat, args, *kw)

_orig_comp = openai.Completion.create
openai.Completion.create = lambda args, *kw: openai_with_backoff(_orig_comp, args, *kw)
```

  1. Expose max_retries/initial_delay in user config or CLI flags.
  2. Add unit tests that simulate openai.error.RateLimitError and verify retry logic.

Benefits

  • CLI no longer crashes on intermittent rate limits.
  • Configurable retry policy for power users.
  • Better out‑of‑the‑box resilience under heavy load.

View original on GitHub ↗

7 Comments

Joshuapwilley · 1 year ago

This issue makes this tool unusable in it's current state.

tibo-openai collaborator · 1 year ago

Plan looks good, please send a PR!

etburke · 1 year ago

+1

OneGitHubUsername · 1 year ago
Plan looks good, please send a PR!

I'm just a brain-damaged vibe coder. I had codex write this issue. I'm not confident enough to send a pr. Apologies!

tibo-openai collaborator · 1 year ago

In progress implementation is here:
https://github.com/openai/codex/pull/176

tibo-openai collaborator · 1 year ago

Should be fixed in latest release, please install 0.1.2504221401 and re-open if this is still an issue you encounter.