30k token limit when using API after ChatGPT exhaustion (IDE)

This makes codex completely unusable after you use the ChatGPT allowance.

I contacted support and they just said "This is expected behavior under our API rate limit system, even if you're using your own API key after hitting ChatGPT limits. You’ll need to wait for your quota to reset or reduce the number of tokens sent in each request."

They are completely oblivious as to how useless this makes codex. A typical request on a codebase of less than 1k lines of code already uses 120k tokens, so a 30k token limit means you literally cannot use codex...

the gpt-5-codex model uses way too many tokens you may want to try gpt-5 instead. if your API account is tier 1 you may need to fund it to increase to tier 2-3 to get usable rate limits.

the gpt-5-codex model uses way too many tokens you may want to try gpt-5 instead.

I want to use gpt-5-codex. That is literally the reason I am using codex. GPT‑5-Codex uses 93.7% fewer tokens than GPT‑5: https://openai.com/index/introducing-upgrades-to-codex/

if your API account is tier 1 you may need to fund it to increase to tier 2-3 to get usable rate limits.

You need to preload your account with hundreds of dollars to get to ChatGPT type per-minute limits. I shouldn't need to do that just to continue a service I already pay for. I am happy to pay for my usage, but my rate limits shouldn't be diminished when I want to pay directly for it. This is clearly an oversight in the way the service has been set up

they indeed claimed the gpt-5-codex uses less tokens in the announcement, but there have been many users in the issues here and on twitter who have found the codex model to consume token usage significantly faster than gpt-5 does in practice.

they indeed claimed the gpt-5-codex uses less tokens in the announcement, but there have been many users in the issues here and on twitter who have found the codex model to consume token usage significantly faster than gpt-5 does in practice.

Even gpt-5 consumes a lot.

Something is definetly wrong. (+0 -2), (+6 -7) line changes in 2 files and

⏱️ Usage Limits
  • 5h limit    : [███████             ] 37% used
  • Weekly limit: [████                ] 18% used

raised to:

⏱️ Usage Limits
  • 5h limit    : [█████████           ] 43% used
  • Weekly limit: [████                ] 19% used

Model: gpt-5 high
Plan: Plus

📊 Token Usage
  • Session ID: xxxx
  • Input: 1,057 (+ 5888 cached)
  • Output: 272
  • Total: 1,329

⏱️ Usage Limits
  • 5h limit    : [█████████           ] 44% used
    Resets at: Sep 25, 2025 2:37 PM
  • Weekly limit: [████                ] 20% used
    Resets at: Oct 1, 2025 11:31 AM

  • Input: 1,057 (+ 5888 cached)
  • Output: 272
  • Total: 1,329

Such token usage raised usage limit by 1% on both. Model: gpt-5 medium.

And last week i could have a 1 session with +3M tokens and did not faced either 5h limit or weekly one, and on next day was able to continue with same usage.

That's very strange, because the TPM for Tier 1 was increased to 500k.

That's very strange, because the TPM for Tier 1 was increased to 500k.

It seems this must have changed since I opened this issue (and made a complaint to support). This is what I was getting previously (and the limit was showing 30k on a Tier 1 account):

stream error: stream disconnected before completion: Request too large for gpt-5-codex in organization org-XXXXXXXXXXXXXXX on tokens per min (TPM): Limit 30000, Requested 34723. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.; retrying 2/5 in 433ms…

Now it seems the limit is showing as 500k on my account page. I haven't tested it, as I have some allowance on ChatGPT at the moment. I received an email off support 12 hours ago and the increase wasn't mentioned there - so I guess it has happened since then.

I wont close the issue so we can get some official response on the matter to know if this new tier limit is correct and going to stay.

Codex is definetly buggy with usage, it uses more that needed. I tried to add side model providers (copilot, synthetic) via API and it uses way more than needed with 1 promt.

I'm on Pro+ plan, 1 single fix and my "Premium requests" usage raised from 8.9% to 11.1%

@ViktorsBaikers do you have MCPs that might be adding a lot to context? This has been a common source of people burning through rate limits in the past.

@gpeal No signle MCP at all.

@ViktorsBaikers

In the UI/plan, the system only counts new tokens toward the usage limit.
This masks the real cost because in the API, you pay for the entire context every time, including all old tokens.

Even gpt-5 consumes a lot. Something is definetly wrong. (+0 -2), (+6 -7) line changes in 2 files and

The problem is that for those two small changes, you reloaded the entire previous context.
If the context was 150k tokens, you paid 300k tokens for just those two edits because you ran two full iterations.

With selective handling under something like agent_write,
you would have paid only about 500 tokens instead.

The same thing happens if you read 10 lines of code from two files —
you wouldn’t pay for just 20 lines of code,
you’d end up paying 300k tokens instead.

Let's say the price reduction for cache will amortize it by 4x to 2x, depending on the provider.
In that case, for editing 4 lines of code, you’d pay for around 75,000 tokens.

With an agent_read, you would only pay for those 20 lines, nothing more.

This brutally overloads the GPU, wasting massive resources.

---

Example

You read three files, each 2,000 tokens:

Request 1 → 2,000
Request 2 → 2,000 + 2,000 = 4,000
Request 3 → 2,000 + 2,000 + 2,000 = 6,000
TOTAL = 12,000

UI shows: 6,000 (only new tokens counted)

API billing: 12,000 (full context re-sent every step)

With 10 iterations:

Real usage ≈ 40k tokens

API cost: ≈ 400k tokens

---

Why Cache Explodes

Each new step re-adds all previous data into the cache:

Step 1 → _Cache_ = 1k
Step 2 → _Cache_ = 1k + 1k = 3k
Step 3 → _Cache_ = 1k + 1k + 1k = 6k
Step 4 → _Cache_ = 10k
...

Cache grows quadratically, not linearly.

With a 30k context, a small edit should add _~2%_ to cache,
but here it can jump to 300k+, meaning the same data is reprocessed repeatedly.

If you have 90% context left, which is about 30k tokens in the window,
sending something simple like "ok" should not grow the cache significantly.
Even after 10 prompts, the cache should stay under 100%, around 29k tokens.

However, in this case, after 10 prompts, you see the cache explode:

It grows by the entire previous context each time.

The cache ends up at 1,000%, around 300k tokens, far exceeding the context window.

The cache should never exceed the context window size.
I’m not sure if this directly impacts cost, but it’s a clear sign that something is wrong with cache management.

---

Why Some Tools Avoid This

Systems like Cursor or Claude probably handle context linearly by:

Splitting work into agents:

main – global context

reader – only current fragment

writer – applies changes

Using snapshots outside the main context, so past data isn’t reloaded each time.

---

Key Insight

The plan hides the problem by counting only new tokens,
but API billing reflects the real quadratic growth.
To avoid massive costs and cache overload:

Send only deltas, never the full history each step.

@gpeal

The problem of excessive overcounting (7–9×) was never caused by MCP,
but by including the entire previous context in every request.
Counting only new tokens, as it’s done now, is just artificial masking of the real issue.

The flickering state of MCP or reloading could only cause the cache to miss updates due to changes,
but these are microscopic losses at this scale.

That said, in modern systems, the cache should be granular for tools and context,
as supporting dynamic behavior is already a standard practice in computing systems —
similar to how .so files, JIT compilation, and other dynamic mechanisms work.

URL

[https://www.anthropic.com/engineering/writing-tools-for-agents](https://www.anthropic.com/engineering/writing-tools-for-agents
)

However, if an LLM agent uses a tool that returns ALL contacts and then has to read through each one token-by-token, it's wasting its limited context space on irrelevant information (imagine searching for a contact in your address book by reading each page from top-to-bottom—that is, via brute-force search). The better and more natural approach (for agents and humans alike) is to skip to the relevant page first (perhaps finding it alphabetically).

We recommend building a few thoughtful tools targeting specific high-impact workflows, which match your evaluation tasks and scaling up from there. In the address book case, you might choose to implement a search_contacts or message_contact tool instead of a list_contacts tool.

Through the iterative, evaluation-driven process we’ve described in this post, we've identified consistent patterns in what makes tools successful: Effective tools are intentionally and clearly defined, use agent context judiciously, can be combined together in diverse workflows, and enable agents to intuitively solve real-world tasks.

Tools can consolidate functionality, handling potentially multiple discrete operations (or API calls) under the hood. For example, tools can enrich tool responses with related metadata or handle frequently chained, multi-step tasks in a single tool call.

My option:

Therefore, when a model needs to analyze multiple files at once, it’s better to load all four in parallel to avoid duplication and then return a consolidated snapshot to the main context. If the max context limit is a concern, each file can be processed in a separate chat and the four snapshots merged afterward.

Or you could use a side agent to read even 10 files of 100k tokens each, but do it iteratively by loading only the current 10k at a time and storing snapshots in the main context about where everything is located.

I wrote about this 1 over months ago,
but it seems my advice is somehow being treated negatively for reasons unknown to me.

I wrote about this, and it looks like Anthropic is thinking along similar lines,
or perhaps they were even inspired to write their article.
It’s also possible they were simply deliberating over a similar problem to the one I described.

The best part is that for the article Anthropic later published,
I actually got a two downvote. :)
Topic

We have found selecting between prefix- and suffix-based namespacing to have non-trivial effects on our tool-use evaluations. Effects vary by LLM and we encourage you to choose a naming scheme according to your own evaluations.

I have an irresistible feeling that I discovered this a few posts below in this thread.

<img width="840" height="78" alt="Image" src="https://github.com/user-attachments/assets/ccc2f3ac-62e3-404a-b529-d90b7a807e14" />

Putting cost aside, this will dramatically improve quality.
A model tasked with finding a needle in a haystack or making a precise change while operating with a bloated context of around 200k tokens (teetering on the edge of a mirage) will be far less effective than an agent with virtually zero context, focused solely on executing a single instruction.
In this matchup, the agent will be vastly more precise.
I also added an issue with a proposal to use the File Search API.

Context-Engineering

"Providing “cognitive tools” to GPT-4.1 increases its pass@1 performance on AIME2024 from 26.7% to 43.3%, bringing it very close to the performance of o1-preview." — IBM Zurich

The attention mechanism also operates quadratically, so each additional token adds extra cost —
and not just by one token, but by the square of the total number of tokens.

@makeable
Regarding the limit issue, this is official.

Best regards.

@AdsQnn The missing factor is that when there is a matching token prefix, the tokens are cached and don't count toward your rate limit.

I understand that there is still some confusion and open questions about rate limits, what counts, etc. We're going to continue to iterate here and we want to be as transparent as possible (like the new rate limit UI that launched this week.

I'm going to close this out because this issue, specifically, isn't actionable on its own but we hear you.

@AdsQnn The missing factor is that when there is a matching token prefix, the tokens are cached and don't count toward your rate limit.

@gpeal

In the UI/plan, the system only counts new tokens toward the usage limit.

This masks the real cost because in the API, you pay for the entire context every time, including all old tokens.

And regarding the API mentioned in the topic — are only new tokens counted there as well?
Is the cache free? Or, for example, is 8 million tokens of cache usage across several sessions just as expensive as 500,000 tokens of cache usage? How does this scale?

I fully understand — and have described — that only new tokens count toward the throughput limit, which is why this behavior is problematic and somewhat masks what’s really happening.
You see, in the API all tokens are included in the cost, not just the ones counted toward the throughput limit (I know they are priced cheaper).
I also pointed out that even with the 4× amortization of cache pricing, the overall cost in the API will still be high given the agent’s behavior.

On the other hand, even though cached tokens don’t count toward the throughput limit, are they still neutral for GPU load?
Or am I mistaken here?

If the overall GPU load were to decrease while keeping the same level of utilization, wouldn’t that potentially make things more efficient?

Secondly, the model also loses performance as the context grows.
Anthropic even mentioned this in their article about context engineering, so it seems achievable — not necessarily right away, of course. :)

I hope what I’ve written isn’t interpreted negatively as criticism.
I just thought it would be worth analyzing this carefully.

I’ll try to hold back from sharing such detailed opinions, so no worries. :)

Thank you for your response, and I wish you the best of luck and great success in your work on Codex!

@AdsQnn You can read about our prompt caching here and we aim to keep Codex as cache-efficient as possible.

Do not this line from the docs:

Cached prefixes generally remain active for 5 to 10 minutes of inactivity. However, during off-peak periods, caches may persist for up to one hour.

@gpeal
Ehh, I get the feeling we’re not quite understanding each other. I know this topic very well and have read this and many other sources in great detail.
But what you decide to do is entirely up to you — I’ve described the problem thoroughly, and it’s 100% real.

What I’m describing is backed by numerous studies, and I’ve included links to them.
You’ll remember what I said at some point down the road. :)

No offense, of course.

"Context engineering is the delicate art and science of filling the context window with just the right information for the next step." — Andrej Karpathy. A frontier, first-principles handbook inspired by Karpathy and 3Blue1Brown for moving beyond prompt engineering to the wider discipline of context design, orchestration, and optimization.

You’re talking about cache, which reduces usage by 10×, while I’m talking about a contexting technique that reduces computational complexity by 50×.

I really know what I’m talking about — you might doubt it, and that’s fine, but I’m not going to try to convince anyone by force. :)
Please don’t take this the wrong way.

What you’re saying and what I’m saying don’t have to be mutually exclusive at all — both can coexist, and that’s when the real savings happen.

Besides, you can always ask Codex if I’m right. :)

If you have 200k tokens in the main context and you start searching for which file contains method xyz — say, over 10 iterations — do you really think it will be just as fast and cheap as when the main agent delegates the task to an agent_search?

The agent_search would load only the current file into context each step until it finds the method, then return to the main agent with:

"Here it is — file.r, line 220."

In this approach,

you pay only for the exact number of new tokens,

and pay 0 for the cache,
instead of repeatedly reloading the entire 200k-token context every single iteration.

The number of new tokens is the same in both approaches.

But with agent_search, the cached tokens reloaded = 0 (because it only loads the current file each step).

With the current approach, keeping 200k tokens in main context × 10 iterations to find xyz yields ~2,000,000 cached tokens reprocessed.

Since cache is ~10× cheaper, that still bills like ~200k tokens of “effective” cost.

So to find a single fragment:
current (monolithic) search ≈ agent_search + ~2M cache → ~200k effective tokens,
whereas pure agent_search pays only for the true new tokens and avoids cache bloat.

Simply, with every pass — think → read, even if it's just one line,
you will again pay:

200,000 cache 0.125 + new_context 1.25

Which simplifies to:

210,000 0.125 cache + new_context 1.25

After 10 iterations, the cost becomes:

2,000,000 cache 0.125 + 20,000 new_context 1.25
The problem is that you're trying to forget about that 2,000,000 × 0.125.

---

Now multiply those 2M tokens × 50 questions in a session:

2,000,000 × 50 = 100,000,000

That’s 100 million tokens total.

Dividing by 10 (because cache is ~10× cheaper):

100,000,000 ÷ 10 = 10,000,000

So you end up with 10 million effective tokens just from cache overhead.

If $0.125 is the cost per million tokens cache:

1. First, divide 100M tokens by 1M:

100,000,000 ÷ 1,000,000 = 100

1. Then multiply by $0.125:

100 × 0.125 = 12.5

The total additional cost is $12.50,
but it could have been $0.

This is only the extra cost incurred during one ordinary session.

What I mainly mean is GPU usage, which directly impacts how you set the limits.

Cached prefixes generally remain active for 5 to 10 minutes of inactivity. However, during off-peak periods, caches may persist for up to one hour.

The fact that the cache persists for an hour doesn’t change the reality that
you pay for the entire context window every single time —
just 10× less per token.

But you’re still paying, every single time.

The difference is that during that one hour,
you'll pay $0.125 per million tokens instead of $1.25.

But if those extra tokens add up to 100 million,
you'll still end up paying a lot.

#1 Set Up a Search Engine: Use a public search API, like Google's Custom Search API, to perform web searches and obtain a list of relevant search results.

#2 Build a Search Dictionary: Collect the title, URL, and a summary of each web page from the search results to create a structured dictionary of information.
#3. Generate a RAG Response: Implement Retrieval-Augmented Generation (RAG) by passing the gathered information to the LLM, which then generates a final response to the user's query.

I’m talking about exactly the same thing.

[Building a Bring Your Own Browser (BYOB) Tool for Web Browsing and Summarization](https://cookbook.openai.com/examples/third_party/web_search_with_google_api_bring_your_own_browser_tool
)
I’m talking specifically about files here —
this applies directly to your Cookbook.

Why flip through every single page in full, one by one,
using the main context like Codex currently does?

Isn't there cache for that?

Step 3: Set up Agentic Orchestration to run the application

There are various ways to orchestrate the Agents based on the application requirements. In this example, we will use a simple orchestration where the user provides a task and the agents are called in sequence to accomplish the task

https://cookbook.openai.com/examples/object_oriented_agentic_approach/secure_code_interpreter_tool_for_llm_agents#step-3-set-up-agentic-orchestration-to-run-the-application

@AdsQnn The missing factor is that when there is a matching token prefix, the tokens are cached and don't count toward your rate limit.

But you set the limits based on GPU usage. And the current system makes them run at least 10x harder than necessary. If you applied these fixes, you could increase the limits because GPU usage would drop.

So if you say, "we set limits based on new tokens," you're creating an illusion for yourself. You always pay for GPU usage in the end. The cost of a token is calculated based on the computational power required to process it.

When you offer cache as if it were "free," you fail to see that you're unconsciously increasing the cost of the new tokens. This is because the final cost will always equal the total energy consumed. And unfortunately, cache is never truly neutral either.

@gpeal

Second, they help manage context: subagents use their own isolated context windows, and only send relevant information back to the orchestrator, rather than their full context.

[https://www.anthropic.com/engineering/building-agents-with-the-claude-agent-sdk](https://www.anthropic.com/engineering/building-agents-with-the-claude-agent-sdk
)
Alright, I’ll stop spamming now. :)

Edit

In a 100-turn web search evaluation, context editing enabled agents to complete workflows that would otherwise fail due to context exhaustion—while reducing token consumption by 84%.

https://www.anthropic.com/news/context-management

[https://github.com/anthropics/claude-cookbooks/blob/main/tool_use/memory_cookbook.ipynb](https://github.com/anthropics/claude-cookbooks/blob/main/tool_use/memory_cookbook.ipynb
)
Cheers !