Codex CLI TUI drops keyboard input in RunPod/JupyterLab terminal; raw /dev/tty reader fixes input loss
What version of Codex CLI is running?
0.125.0
What subscription do you have?
ChatGPT Plus
Which model were you using?
gpt-5.5
What platform is your computer?
Remote environment: Ubuntu 22.04.5 LTS on RunPod container/pod Client machine: Windows 10, using Chrome browser to access RunPod JupyterLab terminal
What terminal emulator and version are you using (if applicable)?
RunPod JupyterLab terminal in Chrome browser, via RunPod proxy URL. The terminal appears to be the JupyterLab web terminal / xterm.js style terminal.
What issue are you seeing?
Codex CLI TUI v0.125.0 starts normally in a RunPod/JupyterLab web terminal, but keyboard input becomes unreliable. When typing into the Codex prompt, only some characters reach the TUI/event stream.
Example: typing abc123 sometimes resulted in only a partial sequence being received, such as:
RAW KEY CHARS: as
COUNT: 2
This made the TUI effectively unusable.
This does not appear to be a general terminal-byte-delivery problem. A direct raw /dev/tty probe in the same terminal receives the full input correctly:
BYTES: [97] CHARS: 'a'
BYTES: [98] CHARS: 'b'
BYTES: [99] CHARS: 'c'
BYTES: [49] CHARS: '1'
BYTES: [50] CHARS: '2'
BYTES: [51] CHARS: '3'
BYTES: [3] CHARS: '\x03'
After testing several mitigations, the reliable fix/workaround was a hard-exclusive raw /dev/tty keyboard reader inside Codex, bypassing the normal crossterm keyboard polling path for keyboard input in this affected environment.
With that patched path active, Codex receives keyboard input reliably, including ASCII, Enter, Backspace, Ctrl+C, spaces, and UTF-8 characters such as ö, ü, ä, and ß.
Successful patched-run markers:
event_stream.keyboard_source decision=raw_dev_tty_hard_exclusive
raw_stdin_keyboard.spawn active=true
raw_stdin_keyboard.open_dev_tty ok=true
raw_stdin_keyboard.source=/dev/tty
event_stream.crossterm_poll_skipped_raw_stdin_keyboard
Successful input evidence:
raw_stdin_keyboard.recv bytes=[97] hex=[0x61]
raw_stdin_keyboard.emit code=Char char='a' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[98] hex=[0x62]
raw_stdin_keyboard.emit code=Char char='b' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[99] hex=[0x63]
raw_stdin_keyboard.emit code=Char char='c' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[49] hex=[0x31]
raw_stdin_keyboard.emit code=Char char='1' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[50] hex=[0x32]
raw_stdin_keyboard.emit code=Char char='2' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[51] hex=[0x33]
raw_stdin_keyboard.emit code=Char char='3' modifiers=KeyModifiers(0x0)
UTF-8 evidence after buffering/decoding multibyte sequences:
raw_stdin_keyboard.recv bytes=[195, 182] hex=[0xc3, 0xb6]
raw_stdin_keyboard.emit code=Char char='ö' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[195, 188] hex=[0xc3, 0xbc]
raw_stdin_keyboard.emit code=Char char='ü' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[195, 164] hex=[0xc3, 0xa4]
raw_stdin_keyboard.emit code=Char char='ä' modifiers=KeyModifiers(0x0)
raw_stdin_keyboard.recv bytes=[195, 159] hex=[0xc3, 0x9f]
raw_stdin_keyboard.emit code=Char char='ß' modifiers=KeyModifiers(0x0)
So the issue appears to be in the normal Codex TUI/crossterm keyboard input path in this terminal environment, not in the terminal’s ability to deliver bytes.
What steps can reproduce the bug?
1.Start a RunPod pod/container with Ubuntu 22.04.5 LTS.
2.Open the RunPod JupyterLab web terminal through the RunPod browser/proxy interface.
3.Start Codex CLI v0.125.0 in the terminal:
codex
4.Wait until the Codex TUI renders.
5.Type a simple test string into the Codex prompt, for example:
abc123
6.Observe that the Codex prompt/TUI may receive only part of the typed characters, or input becomes unreliable/unusable.
During debugging, I used an instrumented Codex build that logged raw crossterm key events. In failing runs, the received key sequence was incomplete, for example:
RAW KEY CHARS: as
COUNT: 2
As a control test, a direct raw /dev/tty probe in the same terminal receives the full abc123 sequence correctly.
Python probe used:
import os, termios, tty
fd = os.open("/dev/tty", os.O_RDWR)
old = termios.tcgetattr(fd)
try:
tty.setraw(fd)
os.write(fd, b"RAW BYTE PROBE active. Type exactly abc123, then Ctrl+C.\r\n")
while True:
b = os.read(fd, 16)
msg = f"\r\nBYTES: {list(b)} CHARS: {repr(b.decode('utf-8', 'replace'))}\r\n"
os.write(fd, msg.encode())
if 3 in b:
os.write(fd, b"STOP\r\n")
break
finally:
termios.tcsetattr(fd, termios.TCSADRAIN, old)
os.close(fd)
That probe receives all characters correctly:
BYTES: [97] CHARS: 'a'
BYTES: [98] CHARS: 'b'
BYTES: [99] CHARS: 'c'
BYTES: [49] CHARS: '1'
BYTES: [50] CHARS: '2'
BYTES: [51] CHARS: '3'
BYTES: [3] CHARS: '\x03'
Mitigations tested that did not fully fix the problem:
disabling cursor position queries
disabling foreground/background color queries
disabling terminal palette re-query
throttling crossterm polling by 15 ms
internal raw probes at several TUI lifecycle stages
raw stdin reader without hard-exclusive crossterm keyboard bypass
The working approach was a hard-exclusive raw /dev/tty keyboard reader that bypasses the normal crossterm keyboard polling path for keyboard input in this affected environment.
What is the expected behavior?
Codex CLI TUI should reliably receive all typed keyboard input in the RunPod/JupyterLab web terminal.
For example, typing:
abc123
should result in exactly those characters appearing in the Codex prompt.
The TUI should also correctly handle normal editing/control input such as:
Enter
Backspace
Ctrl+C
spaces
UTF-8 characters such as ö, ü, ä, and ß
The behavior should match the direct raw /dev/tty probe, which receives all bytes correctly in the same terminal.
Additional information
This was investigated with an instrumented local build of Codex CLI v0.125.0.
Important findings:
The terminal itself can deliver all bytes correctly through /dev/tty.
Codex startup and rendering work.
The normal Codex TUI/crossterm keyboard path receives only partial keyboard input in this environment.
Disabling terminal cursor/color queries did not fully fix the issue.
Poll throttling did not fully fix the issue.
A hard-exclusive raw /dev/tty keyboard reader did fix input in the affected environment.
UTF-8 input works once multibyte sequences are buffered and decoded before emitting KeyCode::Char.
Relevant successful patched-run markers:
safe_terminal_modes active=true
safe_terminal_modes disable_focus_change sent
safe_terminal_modes disable_bracketed_paste sent
safe_terminal_modes disable_mouse_capture sent
safe_terminal_modes disable_keyboard_enhancement sent
custom_terminal.cursor_query_disabled_by_env
terminal_palette.query_foreground_color disabled_by_env
terminal_palette.query_background_color disabled_by_env
event_stream.keyboard_source decision=raw_dev_tty_hard_exclusive
raw_stdin_keyboard.spawn active=true
raw_stdin_keyboard.open_dev_tty ok=true
raw_stdin_keyboard.source=/dev/tty
event_stream.crossterm_poll_skipped_raw_stdin_keyboard
Current root-cause hypothesis:
The issue is likely caused by the normal crossterm event/input path interacting badly with the RunPod/JupyterLab web terminal environment and/or competing reads/terminal queries against the controlling TTY.
Potential fix direction:
Add a fallback keyboard backend for affected terminal environments:
detect or opt into a raw /dev/tty keyboard reader
avoid simultaneous crossterm keyboard polling when that backend is active
keep crossterm for rendering / non-key events where appropriate
decode UTF-8 multibyte input correctly
avoid terminal-interactive git credential prompts stealing or contending for /dev/tty
I am not opening a PR because the contributing guidelines say unsolicited PRs are not accepted. I can provide a minimal patch/log bundle if helpful.
4 Comments
Small follow-up after further testing:
The raw
/dev/ttykeyboard fallback fixed normal typing, ASCII, Enter, Backspace, Ctrl+C, and UTF-8 input, but it initially broke large multiline paste behavior because it bypassed crossterm's normal bracketed-paste path.The follow-up fix was to preserve/re-enable bracketed paste handling in the raw keyboard backend:
ESC [ 200 ~as bracketed paste startESC [ 201 ~as bracketed paste endTuiEvent::Paste(String)instead of feeding the pasted content only as individual key chars/dev/ttykeyboard input active so crossterm does not compete for keyboard bytesAfter that, normal keyboard input, UTF-8 input (
ö ü ä ß), and long multiline prompts all work reliably in the RunPod/JupyterLab terminal.So the likely upstream fix direction is not only "read keyboard bytes from
/dev/ttyin this environment", but also "preserve bracketed paste semantics when using that fallback backend."@Leon102938 thanks for the detailed report and the evidence bundle. The user-facing symptom looks real, but the attached bundle appears to point to a different strongest cause than the issue body: README_EVIDENCE.md says the strongest hypothesis was a competing /dev/tty reader during startup, especially an interactive Git credential prompt, and that stable behavior required both a clean non-git directory and hard-disabled Git prompts.
Could you share:
GIT_TERMINAL_PROMPT=0 GCM_INTERACTIVE=never GIT_ASKPASS=/bin/false SSH_ASKPASS=/bin/false codexThat would help distinguish a Codex/crossterm input bug from another startup process temporarily stealing the controlling TTY.
Thanks, I ran the requested isolation matrix in a fresh RunPod pod with the original
codexbinary, not my patchedcodex-fixedwrapper.Environment:
/workspace/tmp/codex_clean_non_git/usr/bin/codexcodex-cli 0.125.0abc123Version/context check:
command -v codex->/usr/bin/codexcodex --version->codex-cli 0.125.0pwd->/workspacegit rev-parse --is-inside-work-tree->trueMatrix results:
cd /workspace && codexabc123resulted in onlyc2reaching/appearing in the Codex TUI.cd /workspace && GIT_TERMINAL_PROMPT=0 GCM_INTERACTIVE=never GIT_ASKPASS=/bin/false SSH_ASKPASS=/bin/false codexabc123worked normally.cd /tmp/codex_clean_non_git && codexabc123worked normally.cd /tmp/codex_clean_non_git && GIT_TERMINAL_PROMPT=0 GCM_INTERACTIVE=never GIT_ASKPASS=/bin/false SSH_ASKPASS=/bin/false codexabc123worked normally.Additional pre-auth observation:
Before completing login, arrow-key navigation in the original Codex login/auth TUI was also unreliable in the RunPod/JupyterLab terminal. I had to press arrow keys many times before the highlighted selection moved. This was with the original Codex binary.
Based on this matrix, the issue now looks strongly tied to running original Codex inside the Git working tree with Git credential/interactive prompts enabled. Disabling Git terminal prompts fixes the input loss in
/workspace, and the issue does not reproduce in a clean non-git directory.Correction / clarification about my earlier workaround:
The raw
/dev/ttyfallback I tested was useful as a diagnostic workaround, but based on the isolation matrix above I no longer think it is necessarily the right root-cause fix. The results point much more strongly to interactive Git/GCM credential prompts, or another startup process, temporarily stealing or contending for the controlling TTY while the Codex TUI is active.So the cleaner upstream direction may be to make sure Codex prevents or isolates interactive Git credential prompts during TUI startup, instead of needing a full replacement keyboard backend.
Related Crossterm/input-consumption investigation: https://github.com/openai/codex/issues/24527. Its new non-consuming readiness probe can distinguish input already waiting on the PTY from later TUI handling; one initialization smoke captured a 2.5-second readiness-to-mapping interval. No dropped-key behavior like this RunPod case has been proven there, so the environment-specific raw-tty finding remains distinct.