let user customize context_window
Resolved 💬 2 comments Opened Sep 12, 2025 by jiangying000 Closed Dec 14, 2025
What feature would you like to see?
I believe most users will never use up the 128k max output token (e.g. I almost never use>30k for output), but usually run out of the context window after long chat. Can we make the context window parameters customizable and calculate max_output_tokens based on the context_window ?
Are you interested in implementing this feature?
Not now
Additional information
Maybe it's better to let users specify the max output token and calculate the context window internally.
I get this error almost every day because of context not enough, which is annoying :
stream error: stream disconnected before completion: Your input exceeds the context window of this model. Please adjust your input and try again.
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗