codex throttles local inference engines to 8 bursts per hour on long multi-step requests
What version of Codex is running?
0.39.0
Which model were you using?
gpt-osss
What platform is your computer?
amd64 linux for cocex with aarch64 macos for ollama
What steps can reproduce the bug?
Create prompt that will induce multiple background inferrence requests.
Watch GPU only being scheduled up to 8 times per hour. Usually looks something like 1.0 minute of GPU at 100% then idle for 6.5 minutes - or until the next 1/8 hour slot arrives.
What is the expected behavior?
no idle time ( )
What do you see instead?
we see CPU increase - but no GPU activity unless it is on a 1/8 hour mark with 00:00 starting at the first inference/GPU request.
Additional information
It _feels_ like when I first start a session, this isn't happening for the first couple of prompts ( good steady GPU usage, one inference call after another ) and that somehow it loses track of the state it should be in ( as if codex is waiting, assuming the inference provider is busy - or codex doesn't know that the provider is ready to accept the next request ) based on some sort of feedback codex may be be expecting and has not received from the provider? Maybe some sort of exponential timeout backoff is related - perhaps before some sort of backoff limit or timeout is reached and codex ( not knowing it could have proceeded sooner) goes ahead with the next action in its ongoing plan?
6 Comments
!image
showing 30 minute (4/8ths) run, with next burst starting exactly 7 minutes and 30 seconds (throttling to 8 per hour) from the beginning of the previous GPU burst.
during the dead time, codex TUI displays the "Working" text effect with the runtime
I had a similar experience when I tried using Codex with GPT-OSS via Ollama (on macOS 26 Apple Silicon) but I figured it was an issue with Ollama's inference since I was seeing Ollama use 1100% CPU in between the periodic GPU usage spikes.
You're seeing 0% CPU usage from Ollama in your case?
oh, interesting... yes, it does seem to show higher cpu when the gpu drops off
i wonder if this is a context size spillage issue.. maybe i need to create a Modelfile'd tweak of the base gpt-oss model with a hard set context size of 128k. my provider instance has a max size set to 256k for a set of models with that size, which might be allowing it to get squirrely... but, if any of this is related, my assumption has been that the model's own internal trained context size limits how much it might _try_ to use. i also set a 128k limit for it in my codex config.toml profile for gpt-oss
pause, that is what the issue was ( and wow, keeping a tight funnel on the context to stay in-gpu is resulting in a huge performance increase ). closing this issue.
@neurostream Ollama just released an update to their memory management engine that looks promising, I haven't tried it myself yet but they mentioned gpt-oss uses the new features: https://ollama.com/blog/new-model-scheduling