›_ Codex Issues
Codex Issues / #1883

LM Studio OSS

Resolved 💬 3 comments Opened Aug 6, 2025 by tbiens Closed Oct 1, 2025

What is the type of issue?

Documentation is missing

What is the issue?

I was unable to get codex to connect with lm studio.

export CODEX_OSS_BASE_URL="http://localhost:1234/v1"

codex --oss -m "openai/gpt-oss-20b"

LM studio sees the connection attempt but it fails.

Pulling model openai/gpt-oss-20b...Error: OSS setup failed: Pull stream ended unexpectedly without success.

Asking chatgpt for assistance, it had me adjusting config.toml and no luck neither.

Where did you find it?

_No response_

View original on GitHub ↗

3 Comments

DocMAX · 1 year ago

so is it fixed?

DocMAX · 12 months ago

CODEX_OSS_BASE_URL="http://localhost:1234/v1" codex

with config:

model_provider = "oss"
model = "openai/gpt-oss-20b"
approval_policy = "never"
sandbox_mode = "danger-full-access"

[model_providers.oss]
name = "lmstudio"
base_url = "http://localhost:1234/v1"

kind of works...

jschmdt · 11 months ago

This issue is stil not fixed… and a proper documentation is missing.

CODEX_OSS_BASE_URL="http://localhost:1234/v1" codex with config: `` model_provider = "oss" model = "openai/gpt-oss-20b" approval_policy = "never" sandbox_mode = "danger-full-access" [model_providers.oss] name = "lmstudio" base_url = "http://localhost:1234/v1" `` kind of works...

With this config from @DocMAX it still fails for me (macOS 26 Tahoe & LMStudio). Here's a screenshot confirming LM Studio uses the "openai/gpt-oss-20b":

<img width="381" height="169" alt="Image" src="https://github.com/user-attachments/assets/75e2e0d3-75ce-443f-9381-3684482e92e2" />

It starts generating but then crashes…

<img width="1337" height="121" alt="Image" src="https://github.com/user-attachments/assets/dcb732b8-4d82-49bc-b7bb-9ad956c02eee" />

Here's the log from LM Studio:

2025-10-01 15:48:14 [DEBUG]
 Received request: POST to /v1/chat/completions with body [Hidden]
2025-10-01 15:48:14  [INFO]
 [JIT] Requested model (openai/gpt-oss-20b) is not loaded. Loading "openai/gpt-oss-20b" now...
2025-10-01 15:48:14 [DEBUG]
 [LM Studio] GPU Configuration:
  Strategy: evenly
  Priority: []
  Disabled GPUs: []
  Limit weight offload to dedicated GPU Memory: OFF
  Offload KV Cache to GPU: ON
2025-10-01 15:48:14 [DEBUG]
 [LM Studio] Live GPU memory info:
No live GPU info available
2025-10-01 15:48:14 [DEBUG]
 [LM Studio] Model load size estimate with raw num offload layers 'max' and context length '131072':
  Model: 11.96 GB
  Context: 5.91 GB
  Total: 17.87 GB
2025-10-01 15:48:14 [DEBUG]
 [LM Studio] Not using full context length for VRAM overflow calculations due to single GPU setup. Instead, using '8192' as context length for the calculation. Original context length: '131072'.
[LM Studio] Strict GPU VRAM cap is OFF: GPU offload layers will not be checked for adjustment
2025-10-01 15:48:14 [DEBUG]
 [LM Studio] Resolved GPU config options:
  Num Offload Layers: max
  Num CPU Expert Layers: 0
  Main GPU: 0
  Tensor Split: [0]
  Disabled GPUs: []
2025-10-01 15:48:14 [DEBUG]
 Metal : CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | REPACK = 1 |
2025-10-01 15:48:14 [DEBUG]
 llama_model_load_from_file_impl: using device Metal (Apple M3 Max) - 110100 MiB free
llama_model_loader: ------------------------ Adding override for key 'gpt-oss.expert_used_count'
2025-10-01 15:48:14 [DEBUG]
 llama_model_loader: loaded meta data with 33 key-value pairs and 459 tensors from /Users/[USERNAME]/.lmstudio/models/lmstudio-community/gpt-oss-20b-GGUF/gpt-oss-20b-MXFP4.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = gpt-oss
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Openai_Gpt Oss 20b
llama_model_loader: - kv   3:                           general.basename str              = openai_gpt-oss
llama_model_loader: - kv   4:                         general.size_label str              = 20B
llama_model_loader: - kv   5:                        gpt-oss.block_count u32              = 24
llama_model_loader: - kv   6:                     gpt-oss.context_length u32              = 131072
llama_model_loader: - kv   7:                   gpt-oss.embedding_length u32              = 2880
llama_model_loader: - kv   8:                gpt-oss.feed_forward_length u32              = 2880
llama_model_loader: - kv   9:               gpt-oss.attention.head_count u32              = 64
llama_model_loader: - kv  10:            gpt-oss.attention.head_count_kv u32              = 8
llama_model_loader: - kv  11:                     gpt-oss.rope.freq_base f32              = 150000.000000
llama_model_loader: - kv  12:   gpt-oss.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  13:                       gpt-oss.expert_count u32              = 32
2025-10-01 15:48:14 [DEBUG]
 llama_model_loader: - kv  14:                  gpt-oss.expert_used_count u32              = 4
llama_model_loader: - kv  15:               gpt-oss.attention.key_length u32              = 64
llama_model_loader: - kv  16:             gpt-oss.attention.value_length u32              = 64
llama_model_loader: - kv  17:           gpt-oss.attention.sliding_window u32              = 128
llama_model_loader: - kv  18:         gpt-oss.expert_feed_forward_length u32              = 2880
llama_model_loader: - kv  19:                  gpt-oss.rope.scaling.type str              = yarn
llama_model_loader: - kv  20:                gpt-oss.rope.scaling.factor f32              = 32.000000
llama_model_loader: - kv  21: gpt-oss.rope.scaling.original_context_length u32              = 4096
llama_model_loader: - kv  22:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  23:                         tokenizer.ggml.pre str              = gpt-4o
2025-10-01 15:48:14 [DEBUG]
 llama_model_loader: - kv  24:                      tokenizer.ggml.tokens arr[str,201088]  = ["!", "\"", "#", "$", "%", "&", "'", ...
2025-10-01 15:48:14 [DEBUG]
 llama_model_loader: - kv  25:                  tokenizer.ggml.token_type arr[i32,201088]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
2025-10-01 15:48:14 [DEBUG]
 llama_model_loader: - kv  26:                      tokenizer.ggml.merges arr[str,446189]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
llama_model_loader: - kv  27:                tokenizer.ggml.bos_token_id u32              = 199998
llama_model_loader: - kv  28:                tokenizer.ggml.eos_token_id u32              = 200002
llama_model_loader: - kv  29:            tokenizer.ggml.padding_token_id u32              = 199999
llama_model_loader: - kv  30:                    tokenizer.chat_template str              = {#-\n  In addition to the normal input...
llama_model_loader: - kv  31:               general.quantization_version u32              = 2
llama_model_loader: - kv  32:                          general.file_type u32              = 38
llama_model_loader: - type  f32:  289 tensors
llama_model_loader: - type q8_0:   98 tensors
llama_model_loader: - type mxfp4:   72 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = MXFP4 MoE
print_info: file size   = 11.27 GiB (4.63 BPW) 
validate_override: Using metadata override (  int) 'gpt-oss.expert_used_count' = 32
load_hparams: ----------------------- n_expert_used = 32
2025-10-01 15:48:15 [DEBUG]
 load: printing all EOG tokens:
load:   - 199999 ('<|endoftext|>')
load:   - 200002 ('<|return|>')
load:   - 200007 ('<|end|>')
load:   - 200012 ('<|call|>')
load: special_eog_ids contains both '<|return|>' and '<|call|>' tokens, removing '<|end|>' token from EOG list
2025-10-01 15:48:15 [DEBUG]
 load: special tokens cache size = 21
2025-10-01 15:48:15 [DEBUG]
 load: token to piece cache size = 1.3332 MB
print_info: arch             = gpt-oss
print_info: vocab_only       = 0
print_info: n_ctx_train      = 131072
print_info: n_embd           = 2880
print_info: n_layer          = 24
print_info: n_head           = 64
print_info: n_head_kv        = 8
print_info: n_rot            = 64
print_info: n_swa            = 128
print_info: is_swa_any       = 1
print_info: n_embd_head_k    = 64
print_info: n_embd_head_v    = 64
print_info: n_gqa            = 8
print_info: n_embd_k_gqa     = 512
print_info: n_embd_v_gqa     = 512
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-05
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 2880
print_info: n_expert         = 32
2025-10-01 15:48:15 [DEBUG]
 print_info: n_expert_used    = 32
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = yarn
print_info: freq_base_train  = 150000.0
print_info: freq_scale_train = 0.03125
print_info: n_ctx_orig_yarn  = 4096
print_info: rope_finetuned   = unknown
print_info: model type       = 20B
print_info: model params     = 20.91 B
print_info: general.name     = Openai_Gpt Oss 20b
print_info: n_ff_exp         = 2880
print_info: vocab type       = BPE
print_info: n_vocab          = 201088
print_info: n_merges         = 446189
print_info: BOS token        = 199998 '<|startoftext|>'
print_info: EOS token        = 200002 '<|return|>'
print_info: EOT token        = 200007 '<|end|>'
print_info: PAD token        = 199999 '<|endoftext|>'
print_info: LF token         = 198 'Ċ'
print_info: EOG token        = 199999 '<|endoftext|>'
print_info: EOG token        = 200002 '<|return|>'
print_info: EOG token        = 200012 '<|call|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
2025-10-01 15:48:15 [DEBUG]
 load_tensors: offloading 24 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 25/25 layers to GPU
load_tensors: Metal_Mapped model buffer size = 11536.19 MiB
load_tensors:   CPU_Mapped model buffer size =   586.82 MiB
2025-10-01 15:48:15 [DEBUG]
 llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 131072
llama_context: n_ctx_per_seq = 131072
llama_context: n_batch       = 512
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = enabled
llama_context: kv_unified    = false
llama_context: freq_base     = 150000.0
llama_context: freq_scale    = 0.03125
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M3 Max
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: picking default device: Apple M3 Max
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_load_library: loading '/Users/[USERNAME]/.lmstudio/extensions/backends/llama.cpp-mac-arm64-apple-metal-advsimd-1.50.2/default.metallib'
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: GPU name:   Apple M3 Max
ggml_metal_init: GPU family: MTLGPUFamilyApple9  (1009)
ggml_metal_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_init: GPU family: MTLGPUFamilyMetal4  (5002)
ggml_metal_init: simdgroup reduction   = true
ggml_metal_init: simdgroup matrix mul. = true
ggml_metal_init: has residency sets    = false
ggml_metal_init: has bfloat            = true
ggml_metal_init: use bfloat            = false
ggml_metal_init: hasUnifiedMemory      = true
ggml_metal_init: recommendedMaxWorkingSetSize  = 115448.73 MB
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_get_rows_bf16                     (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_set_rows_bf16                     (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_mul_mv_bf16_f32                   (not supported)
ggml_metal_init: skipping kernel_mul_mv_bf16_f32_c4                (not supported)
ggml_metal_init: skipping kernel_mul_mv_bf16_f32_1row              (not supported)
ggml_metal_init: skipping kernel_mul_mv_bf16_f32_l4                (not supported)
ggml_metal_init: skipping kernel_mul_mv_bf16_bf16                  (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_mul_mv_id_bf16_f32                (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_mul_mm_bf16_f32                   (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_mul_mm_id_bf16_f16                (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h40           (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h64           (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h80           (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h96           (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h112          (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h128          (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h192          (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_hk192_hv128   (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_h256          (not supported)
ggml_metal_init: skipping kernel_flash_attn_ext_bf16_hk576_hv512   (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_vec_bf16_h64       (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_vec_bf16_h96       (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_vec_bf16_h128      (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_vec_bf16_h192      (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_vec_bf16_hk192_hv128 (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_vec_bf16_h256      (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_flash_attn_ext_vec_bf16_hk576_hv512 (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_cpy_f32_bf16                      (not supported)
2025-10-01 15:48:15 [DEBUG]
 ggml_metal_init: skipping kernel_cpy_bf16_f32                      (not supported)
ggml_metal_init: skipping kernel_cpy_bf16_bf16                     (not supported)
2025-10-01 15:48:15 [DEBUG]
 llama_context:        CPU  output buffer size =     0.77 MiB
llama_kv_cache_iswa: using full-size SWA cache (ref: https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
llama_kv_cache_iswa: creating non-SWA KV cache, size = 131072 cells
2025-10-01 15:48:15 [DEBUG]
 llama_kv_cache:      Metal KV buffer size =  3072.00 MiB
2025-10-01 15:48:15 [DEBUG]
 llama_kv_cache: size = 3072.00 MiB (131072 cells,  12 layers,  1/1 seqs), K (f16): 1536.00 MiB, V (f16): 1536.00 MiB
llama_kv_cache_iswa: creating     SWA KV cache, size = 131072 cells
2025-10-01 15:48:15 [DEBUG]
 llama_kv_cache:      Metal KV buffer size =  3072.00 MiB
2025-10-01 15:48:16 [DEBUG]
 llama_kv_cache: size = 3072.00 MiB (131072 cells,  12 layers,  1/1 seqs), K (f16): 1536.00 MiB, V (f16): 1536.00 MiB
2025-10-01 15:48:16 [DEBUG]
 llama_context:      Metal compute buffer size =  1027.27 MiB
llama_context:        CPU compute buffer size =   517.65 MiB
llama_context: graph nodes  = 2696
llama_context: graph splits = 2
2025-10-01 15:48:16 [DEBUG]
 common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|return|> logit bias = -inf
common_init_from_params: added <|call|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 131072
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
2025-10-01 15:48:16 [DEBUG]
 GgmlThreadpools: llama threadpool init = n_threads = 12
2025-10-01 15:48:16  [INFO]
 [LM STUDIO SERVER] Running chat completion on conversation with 3 messages.
2025-10-01 15:48:16  [INFO]
 [LM STUDIO SERVER] Streaming response...
2025-10-01 15:48:16 [DEBUG]
 Prompt successfully formatted with Harmony.
2025-10-01 15:48:16 [DEBUG]
 Prompt successfully formatted with Harmony.
2025-10-01 15:48:16 [DEBUG]
 Sampling params:	repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
	dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = -1
	top_k = 40, top_p = 0.800, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.800
	mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
Sampling: 
logits -> logit-bias -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist 
Generate: n_ctx = 131072, n_batch = 512, n_predict = -1, n_keep = 6296
2025-10-01 15:48:16 [DEBUG]
 Total prompt tokens: 6297
Prompt tokens to decode: 6297
BeginProcessingPrompt
2025-10-01 15:48:16 [DEBUG]
 llama.cpp abort:3996: missing specialization for ne20 = 32
2025-10-01 15:48:18 [ERROR]
 The model has crashed without additional information. (Exit code: 6). Error Data: n/a, Additional Data: n/a