Using codex CLI on Linux desktop to connect a LiteLLM/Ollama server, both on a the same LAN, can it work?

Resolved 💬 8 comments Opened May 22, 2025 by fujisan43 Closed Aug 7, 2025

What version of Codex is running?

0.1.2505172129

Which model were you using?

codegeex4

What platform is your computer?

Linux 6.14.6-300.fc42.x86_64 x86_64 unknown

What steps can reproduce the bug?

I have a server and a desktop located on a LAN.

Server IP: 10.0.21.20
Desktop IP: 10.0.21.200

On the server, start a LiteLLM server on port 4000:

litellm --config ~/.litellm/config.yaml

```
$ more ~/.litellm/config.yaml
model_list:

model_name: codegeex4

litellm_params:
model: ollama/codegeex4
api_key: os.environ/OLLAMA_API_KEY

model_name: codellama

litellm_params:
model: ollama/codellama
api_key: os.environ/OLLAMA_API_KEY

model_name: mistral-small3.1

litellm_params:
model: ollama/mistral-small3.1
api_key: os.environ/OLLAMA_API_KEY
``Output of LiteLLM server``
INFO: Started server process [370079]
INFO: Waiting for application startup.

#------------------------------------------------------------#
# #
# 'I get frustrated when the product...' #
# https://github.com/BerriAI/litellm/issues/new #
# #
#------------------------------------------------------------#

Thank you for using LiteLLM! - Krrish & Ishaan

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new

LiteLLM: Proxy initialized with Config, Set models:
codegeex4
codellama
mistral-small3.1
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)
INFO: 10.0.21.200:58308 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:58308 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:58308 - "POST /chat/completions HTTP/1.1" 200 OK
INFO: 10.0.21.200:39788 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:39788 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:44324 - "POST /chat/completions HTTP/1.1" 200 OK
INFO: 10.0.21.200:48680 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:48680 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:48688 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:48688 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:48688 - "POST /chat/completions HTTP/1.1" 200 OK
INFO: 10.0.21.200:40988 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:44878 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:44878 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:56678 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:56678 - "GET /models HTTP/1.1" 200 OK
INFO: 10.0.21.200:50256 - "POST /chat/completions HTTP/1.1" 200 OK
``Every time i make a request in codex, a line likeINFO: 10.0.21.200:50256 - "POST /chat/completions HTTP/1.1" 200 OK`
is printed out.

On the server, start ollama server on port 11434:

systemctl start ollama.service

```
$ more /etc/systemd/system/ollama.service
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"
Environment="OLLAMA_HOST=0.0.0.0:11434"

[Install]
WantedBy=default.target
```

On the desktop,

export OLLAMA_BASE_URL="http://10.0.21.20:4000"

export OLLAMA_API_KEY="sk-1234"

start codex:

codex --model codegeex4 --provider ollama

Then I ask codex:
write python code that reads a binary file
codex responds with
{"name": "shell", "arguments":{"command":["rg","-l","import.",".py"]}}

!Image

So, is it possible for codex to work like this?
What am I missing?

What is the expected behavior?

I expected that codex outputs a small python code that reads a binary file.

What do you see instead?

I get a JSON-style output :
{"name": "shell", "arguments":{"command":["rg","-l","import.",".py"]}}

Additional information

Ouput of journalctl -xeu ollama.service:

May 22 10:06:31 myserver systemd[1]: Started ollama.service - Ollama Service.
░░ Subject: A start job for unit ollama.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit ollama.service has finished successfully.
░░
░░ The job identifier is 4175639.
May 22 10:06:32 myserver ollama[516740]: 2025/05/22 10:06:32 routes.go:1233: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
May 22 10:06:32 myserver ollama[516740]: time=2025-05-22T10:06:32.442+02:00 level=INFO source=images.go:463 msg="total blobs: 16"
May 22 10:06:32 myserver ollama[516740]: time=2025-05-22T10:06:32.442+02:00 level=INFO source=images.go:470 msg="total unused blobs removed: 0"
May 22 10:06:32 myserver ollama[516740]: time=2025-05-22T10:06:32.442+02:00 level=INFO source=routes.go:1300 msg="Listening on [::]:11434 (version 0.6.8)"
May 22 10:06:32 myserver ollama[516740]: time=2025-05-22T10:06:32.452+02:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
May 22 10:06:32 myserver ollama[516740]: time=2025-05-22T10:06:32.722+02:00 level=INFO source=gpu.go:377 msg="no compatible GPUs were discovered"
May 22 10:06:32 myserver ollama[516740]: time=2025-05-22T10:06:32.734+02:00 level=INFO source=types.go:130 msg="inference compute" id=0 library=cpu variant="" compute="" driver=0.0 name="" total="15.5 GiB" available="12.9 GiB"
May 22 10:10:19 myserver ollama[516740]: time=2025-05-22T10:10:19.002+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
May 22 10:10:19 myserver ollama[516740]: time=2025-05-22T10:10:19.064+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
May 22 10:10:19 myserver ollama[516740]: [GIN] 2025/05/22 - 10:10:19 | 200 |  241.013855ms |             ::1 | POST     "/api/show"
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.553+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.576+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.597+02:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.598+02:00 level=INFO source=server.go:106 msg="system memory" total="15.5 GiB" free="12.8 GiB" free_swap="21.8 GiB"
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.598+02:00 level=WARN source=ggml.go:152 msg="key not found" key=chatglm.vision.block_count default=0
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.598+02:00 level=WARN source=ggml.go:152 msg="key not found" key=chatglm.attention.key_length default=128
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.598+02:00 level=WARN source=ggml.go:152 msg="key not found" key=chatglm.attention.value_length default=128
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.599+02:00 level=INFO source=server.go:139 msg=offload library=cpu layers.requested=-1 layers.model=41 layers.offload=0 layers.split="" memory.available="[12.8 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.7 GiB" memory.required.partial="0 B" memory.required.kv="320.0 MiB" memory.required.allocations="[5.7 GiB]" memory.weights.total="4.7 GiB" memory.weights.repeating="4.3 GiB" memory.weights.nonrepeating="485.6 MiB" memory.graph.full="561.0 MiB" memory.graph.partial="789.6 MiB"
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: loaded meta data with 23 key-value pairs and 283 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-816441b33390807d429fbdb1de7e33bb4d569ac68e2203bdbca5d8d79b5c7266 (version GGUF V3 (latest))
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   0:                       general.architecture str              = chatglm
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   1:                               general.name str              = codegeex4-all-9b
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   2:                     chatglm.context_length u32              = 131072
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   3:                   chatglm.embedding_length u32              = 4096
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   4:                chatglm.feed_forward_length u32              = 13696
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   5:                        chatglm.block_count u32              = 40
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   6:               chatglm.attention.head_count u32              = 32
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   7:            chatglm.attention.head_count_kv u32              = 2
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   8:   chatglm.attention.layer_norm_rms_epsilon f32              = 0.000010
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv   9:                          general.file_type u32              = 2
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  10:               chatglm.rope.dimension_count u32              = 64
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  11:               tokenizer.ggml.add_bos_token bool             = false
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  12:                     chatglm.rope.freq_base f32              = 5000000.000000
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  14:                         tokenizer.ggml.pre str              = chatglm-bpe
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,151552]  = ["!", "\"", "#", "$", "%", "&", "'", ...
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,151552]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  17:                      tokenizer.ggml.merges arr[str,151073]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  18:            tokenizer.ggml.padding_token_id u32              = 151329
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  19:                tokenizer.ggml.eos_token_id u32              = 151329
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  20:                tokenizer.ggml.eot_token_id u32              = 151336
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  21:            tokenizer.ggml.unknown_token_id u32              = 151329
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - kv  22:               general.quantization_version u32              = 2
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - type  f32:  121 tensors
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - type q4_0:  161 tensors
May 22 10:11:35 myserver ollama[516740]: llama_model_loader: - type q6_K:    1 tensors
May 22 10:11:35 myserver ollama[516740]: print_info: file format = GGUF V3 (latest)
May 22 10:11:35 myserver ollama[516740]: print_info: file type   = Q4_0
May 22 10:11:35 myserver ollama[516740]: print_info: file size   = 5.08 GiB (4.64 BPW)
May 22 10:11:35 myserver ollama[516740]: load: special_eot_id is not in special_eog_ids - the tokenizer config may be incorrect
May 22 10:11:35 myserver ollama[516740]: load: special tokens cache size = 223
May 22 10:11:35 myserver ollama[516740]: load: token to piece cache size = 0.9732 MB
May 22 10:11:35 myserver ollama[516740]: print_info: arch             = chatglm
May 22 10:11:35 myserver ollama[516740]: print_info: vocab_only       = 1
May 22 10:11:35 myserver ollama[516740]: print_info: model type       = ?B
May 22 10:11:35 myserver ollama[516740]: print_info: model params     = 9.40 B
May 22 10:11:35 myserver ollama[516740]: print_info: general.name     = codegeex4-all-9b
May 22 10:11:35 myserver ollama[516740]: print_info: vocab type       = BPE
May 22 10:11:35 myserver ollama[516740]: print_info: n_vocab          = 151552
May 22 10:11:35 myserver ollama[516740]: print_info: n_merges         = 151073
May 22 10:11:35 myserver ollama[516740]: print_info: EOS token        = 151329 '<|endoftext|>'
May 22 10:11:35 myserver ollama[516740]: print_info: EOT token        = 151336 '<|user|>'
May 22 10:11:35 myserver ollama[516740]: print_info: UNK token        = 151329 '<|endoftext|>'
May 22 10:11:35 myserver ollama[516740]: print_info: PAD token        = 151329 '<|endoftext|>'
May 22 10:11:35 myserver ollama[516740]: print_info: LF token         = 198 'Ċ'
May 22 10:11:35 myserver ollama[516740]: print_info: EOG token        = 151329 '<|endoftext|>'
May 22 10:11:35 myserver ollama[516740]: print_info: EOG token        = 151336 '<|user|>'
May 22 10:11:35 myserver ollama[516740]: print_info: max token length = 1024
May 22 10:11:35 myserver ollama[516740]: llama_model_load: vocab only - skipping tensors
May 22 10:11:35 myserver ollama[516740]: time=2025-05-22T10:11:35.999+02:00 level=INFO source=server.go:410 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-816441b33390807d429fbdb1de7e33bb4d569ac68e2203bdbca5d8d79b5c7266 --ctx-size 8192 --batch-size 512 --threads 6 --no-mmap --parallel 2 --port 35737"
May 22 10:11:36 myserver ollama[516740]: time=2025-05-22T10:11:36.000+02:00 level=INFO source=sched.go:452 msg="loaded runners" count=1
May 22 10:11:36 myserver ollama[516740]: time=2025-05-22T10:11:36.000+02:00 level=INFO source=server.go:589 msg="waiting for llama runner to start responding"
May 22 10:11:36 myserver ollama[516740]: time=2025-05-22T10:11:36.017+02:00 level=INFO source=runner.go:853 msg="starting go runner"
May 22 10:11:36 myserver ollama[516740]: time=2025-05-22T10:11:36.023+02:00 level=INFO source=server.go:623 msg="waiting for server to become available" status="llm server not responding"
May 22 10:11:36 myserver ollama[516740]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-haswell.so
May 22 10:11:36 myserver ollama[516740]: time=2025-05-22T10:11:36.190+02:00 level=INFO source=ggml.go:103 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 compiler=cgo(gcc)
May 22 10:11:36 myserver ollama[516740]: time=2025-05-22T10:11:36.191+02:00 level=INFO source=runner.go:913 msg="Server listening on 127.0.0.1:35737"
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: loaded meta data with 23 key-value pairs and 283 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-816441b33390807d429fbdb1de7e33bb4d569ac68e2203bdbca5d8d79b5c7266 (version GGUF V3 (latest))
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   0:                       general.architecture str              = chatglm
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   1:                               general.name str              = codegeex4-all-9b
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   2:                     chatglm.context_length u32              = 131072
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   3:                   chatglm.embedding_length u32              = 4096
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   4:                chatglm.feed_forward_length u32              = 13696
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   5:                        chatglm.block_count u32              = 40
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   6:               chatglm.attention.head_count u32              = 32
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   7:            chatglm.attention.head_count_kv u32              = 2
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   8:   chatglm.attention.layer_norm_rms_epsilon f32              = 0.000010
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv   9:                          general.file_type u32              = 2
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  10:               chatglm.rope.dimension_count u32              = 64
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  11:               tokenizer.ggml.add_bos_token bool             = false
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  12:                     chatglm.rope.freq_base f32              = 5000000.000000
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  14:                         tokenizer.ggml.pre str              = chatglm-bpe
May 22 10:11:36 myserver ollama[516740]: time=2025-05-22T10:11:36.274+02:00 level=INFO source=server.go:623 msg="waiting for server to become available" status="llm server loading model"
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,151552]  = ["!", "\"", "#", "$", "%", "&", "'", ...
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,151552]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  17:                      tokenizer.ggml.merges arr[str,151073]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  18:            tokenizer.ggml.padding_token_id u32              = 151329
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  19:                tokenizer.ggml.eos_token_id u32              = 151329
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  20:                tokenizer.ggml.eot_token_id u32              = 151336
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  21:            tokenizer.ggml.unknown_token_id u32              = 151329
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - kv  22:               general.quantization_version u32              = 2
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - type  f32:  121 tensors
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - type q4_0:  161 tensors
May 22 10:11:36 myserver ollama[516740]: llama_model_loader: - type q6_K:    1 tensors
May 22 10:11:36 myserver ollama[516740]: print_info: file format = GGUF V3 (latest)
May 22 10:11:36 myserver ollama[516740]: print_info: file type   = Q4_0
May 22 10:11:36 myserver ollama[516740]: print_info: file size   = 5.08 GiB (4.64 BPW)
May 22 10:11:36 myserver ollama[516740]: load: special_eot_id is not in special_eog_ids - the tokenizer config may be incorrect
May 22 10:11:36 myserver ollama[516740]: load: special tokens cache size = 223
May 22 10:11:36 myserver ollama[516740]: load: token to piece cache size = 0.9732 MB
May 22 10:11:36 myserver ollama[516740]: print_info: arch             = chatglm
May 22 10:11:36 myserver ollama[516740]: print_info: vocab_only       = 0
May 22 10:11:36 myserver ollama[516740]: print_info: n_ctx_train      = 131072
May 22 10:11:36 myserver ollama[516740]: print_info: n_embd           = 4096
May 22 10:11:36 myserver ollama[516740]: print_info: n_layer          = 40
May 22 10:11:36 myserver ollama[516740]: print_info: n_head           = 32
May 22 10:11:36 myserver ollama[516740]: print_info: n_head_kv        = 2
May 22 10:11:36 myserver ollama[516740]: print_info: n_rot            = 64
May 22 10:11:36 myserver ollama[516740]: print_info: n_swa            = 0
May 22 10:11:36 myserver ollama[516740]: print_info: n_swa_pattern    = 1
May 22 10:11:36 myserver ollama[516740]: print_info: n_embd_head_k    = 128
May 22 10:11:36 myserver ollama[516740]: print_info: n_embd_head_v    = 128
May 22 10:11:36 myserver ollama[516740]: print_info: n_gqa            = 16
May 22 10:11:36 myserver ollama[516740]: print_info: n_embd_k_gqa     = 256
May 22 10:11:36 myserver ollama[516740]: print_info: n_embd_v_gqa     = 256
May 22 10:11:36 myserver ollama[516740]: print_info: f_norm_eps       = 0.0e+00
May 22 10:11:36 myserver ollama[516740]: print_info: f_norm_rms_eps   = 1.0e-05
May 22 10:11:36 myserver ollama[516740]: print_info: f_clamp_kqv      = 0.0e+00
May 22 10:11:36 myserver ollama[516740]: print_info: f_max_alibi_bias = 0.0e+00
May 22 10:11:36 myserver ollama[516740]: print_info: f_logit_scale    = 0.0e+00
May 22 10:11:36 myserver ollama[516740]: print_info: f_attn_scale     = 0.0e+00
May 22 10:11:36 myserver ollama[516740]: print_info: n_ff             = 13696
May 22 10:11:36 myserver ollama[516740]: print_info: n_expert         = 0
May 22 10:11:36 myserver ollama[516740]: print_info: n_expert_used    = 0
May 22 10:11:36 myserver ollama[516740]: print_info: causal attn      = 1
May 22 10:11:36 myserver ollama[516740]: print_info: pooling type     = 0
May 22 10:11:36 myserver ollama[516740]: print_info: rope type        = 0
May 22 10:11:36 myserver ollama[516740]: print_info: rope scaling     = linear
May 22 10:11:36 myserver ollama[516740]: print_info: freq_base_train  = 5000000.0
May 22 10:11:36 myserver ollama[516740]: print_info: freq_scale_train = 1
May 22 10:11:36 myserver ollama[516740]: print_info: n_ctx_orig_yarn  = 131072
May 22 10:11:36 myserver ollama[516740]: print_info: rope_finetuned   = unknown
May 22 10:11:36 myserver ollama[516740]: print_info: ssm_d_conv       = 0
May 22 10:11:36 myserver ollama[516740]: print_info: ssm_d_inner      = 0
May 22 10:11:36 myserver ollama[516740]: print_info: ssm_d_state      = 0
May 22 10:11:36 myserver ollama[516740]: print_info: ssm_dt_rank      = 0
May 22 10:11:36 myserver ollama[516740]: print_info: ssm_dt_b_c_rms   = 0
May 22 10:11:36 myserver ollama[516740]: print_info: model type       = 9B
May 22 10:11:36 myserver ollama[516740]: print_info: model params     = 9.40 B
May 22 10:11:36 myserver ollama[516740]: print_info: general.name     = codegeex4-all-9b
May 22 10:11:36 myserver ollama[516740]: print_info: vocab type       = BPE
May 22 10:11:36 myserver ollama[516740]: print_info: n_vocab          = 151552
May 22 10:11:36 myserver ollama[516740]: print_info: n_merges         = 151073
May 22 10:11:36 myserver ollama[516740]: print_info: EOS token        = 151329 '<|endoftext|>'
May 22 10:11:36 myserver ollama[516740]: print_info: EOT token        = 151336 '<|user|>'
May 22 10:11:36 myserver ollama[516740]: print_info: UNK token        = 151329 '<|endoftext|>'
May 22 10:11:36 myserver ollama[516740]: print_info: PAD token        = 151329 '<|endoftext|>'
May 22 10:11:36 myserver ollama[516740]: print_info: LF token         = 198 'Ċ'
May 22 10:11:36 myserver ollama[516740]: print_info: EOG token        = 151329 '<|endoftext|>'
May 22 10:11:36 myserver ollama[516740]: print_info: EOG token        = 151336 '<|user|>'
May 22 10:11:36 myserver ollama[516740]: print_info: max token length = 1024
May 22 10:11:36 myserver ollama[516740]: load_tensors: loading model tensors, this can take a while... (mmap = false)
May 22 10:11:36 myserver ollama[516740]: load_tensors:          CPU model buffer size =  5196.84 MiB
May 22 10:11:50 myserver ollama[516740]: llama_context: constructing llama_context
May 22 10:11:50 myserver ollama[516740]: llama_context: n_seq_max     = 2
May 22 10:11:50 myserver ollama[516740]: llama_context: n_ctx         = 8192
May 22 10:11:50 myserver ollama[516740]: llama_context: n_ctx_per_seq = 4096
May 22 10:11:50 myserver ollama[516740]: llama_context: n_batch       = 1024
May 22 10:11:50 myserver ollama[516740]: llama_context: n_ubatch      = 512
May 22 10:11:50 myserver ollama[516740]: llama_context: causal_attn   = 1
May 22 10:11:50 myserver ollama[516740]: llama_context: flash_attn    = 0
May 22 10:11:50 myserver ollama[516740]: llama_context: freq_base     = 5000000.0
May 22 10:11:50 myserver ollama[516740]: llama_context: freq_scale    = 1
May 22 10:11:50 myserver ollama[516740]: llama_context: n_ctx_per_seq (4096) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
May 22 10:11:50 myserver ollama[516740]: llama_context:        CPU  output buffer size =     1.19 MiB
May 22 10:11:50 myserver ollama[516740]: init: kv_size = 8192, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 40, can_shift = 1
May 22 10:11:50 myserver ollama[516740]: init:        CPU KV buffer size =   320.00 MiB
May 22 10:11:50 myserver ollama[516740]: llama_context: KV self size  =  320.00 MiB, K (f16):  160.00 MiB, V (f16):  160.00 MiB
May 22 10:11:50 myserver ollama[516740]: llama_context:        CPU compute buffer size =   561.01 MiB
May 22 10:11:50 myserver ollama[516740]: llama_context: graph nodes  = 1686
May 22 10:11:50 myserver ollama[516740]: llama_context: graph splits = 1
May 22 10:11:51 myserver ollama[516740]: time=2025-05-22T10:11:51.081+02:00 level=INFO source=server.go:628 msg="llama runner started in 15.08 seconds"

View original on GitHub ↗

8 Comments

elasticdotventures · 1 year ago

Fwiw, this probably worked in an earlier release. Try:

npm i -g @openai/codex@0.1.2504301751

fujisan43 · 1 year ago

I succeeded to use codex CLI (version v0.1.2505172129) on my desktop with a remote Ollama server, both on my LAN without LiteLLM.

After installing an Ollama server, you need to set environment variables OLLAMA_HOST and OLLAMA_MODELS.
This can be done in file /etc/systemd/system/ollama.service.

# more /etc/systemd/system/ollama.service 
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_MODELS=/data/ollama/models"

[Install]
WantedBy=default.target

Then restart the ollama service:
# systemctl restart ollama.service

You can also check all the environment variables that have been set with:

# systemctl status ollama.service
May 26 09:42:12 myserver ollama[950632]: 2025/05/26 09:42:12 routes.go:1233: INFO server config 
env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: 
HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:false 
OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 
OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: 
OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 
OLLAMA_MODELS:/data/ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false 
OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost 
https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* 
http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] 
OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"

Then on the desktop, there are a few things to set:

export OLLAMA_BASE_URL=http://myserver:11434/v1
export OLLAMA_HOST=myserver:11434
export OLLAMA_API_KEY=ollama

Note that the base URL is http://myserver:11434/v1 and not http://myserver:11434.

Check the models available as a simple user:

$ ollama list
NAME                                   ID              SIZE      MODIFIED   
mistral-small3.1:latest                b9aaf0c2586a    15 GB     2 days ago    
codegeex4:latest                       867b8e81d038    5.5 GB    2 days ago    
mistral:7b                             f974a74358d6    4.1 GB    3 days ago    
deepseek-r1:8b-llama-distill-q4_K_M    28f8fd6cdc67    4.9 GB    3 days ago    
qwen2.5:7b-instruct-q4_K_M             845dbda0ea48    4.7 GB    3 days ago    
deepseek-r1:7b-qwen-distill-q4_K_M     0a8c26691023    4.7 GB    3 days ago    
llama3.1:8b-instruct-q4_K_M            46e0c10c039e    4.9 GB    3 days ago    
codellama:latest                       8fdf8f752f6e    3.8 GB    3 days ago    
qwen3:1.7b                             458ce03a2187    1.4 GB    3 days ago    
gemma:7b-instruct-v1.1-q4_0            a72c7f4d0a15    5.0 GB    3 days ago    
devstral:24b                           c4b2fa0c33d7    14 GB     3 days ago

Those models are located in /data/ollama/models on the server and not the desktop.

Start the codex CLI with provider ollama and a model:

$ codex --provider ollama --model qwen3:1.7b

And ask something
!Image

Not all model will work. For example, model codegeex4:latest, deepseek-r1:8b-llama-distill-q4_K_M or deepseek-r1:7b-qwen-distill-q4_K_M will give an error like this one:

OpenAI rejected the request. Error details: Status: 400, Code: unknown, Type: api_error, Message: 400 registry.ollama.ai/library/deepseek-r1:8b-llama-distill-q4_K_M does not support tools. Please
    verify your settings and try again.

!Image

Those models do not support tools.

What is the best model that can work like this?

codex-maintainers · 11 months ago

Thanks for the feedback! Please try again on the latest version of Codex CLI. If the issue persists, we'd be happy to take a look at a PR to the codex-rs/ implementation. More details on our project update can be found here.

supermarsx · 10 months ago

If anyone needs as reference, i made a linker to connect to local llms, codex-cli-linker at https://github.com/supermarsx/codex-cli-linker

renesugar · 3 months ago

See the following regarding JSON being displayed instead of the tool being run to complete the task:

https://github.com/openai/codex/discussions/7782

https://github.com/ollama/ollama/issues/13374

https://github.com/ollama/ollama/pull/13351

ACodingfreak · 2 months ago

I am seeing same issue with the latest codex-cli

│ >_ OpenAI Codex (v0.130.0)                        │
│                                                   │
│ model:     ollama/devstral:24b   /model to change │
│ directory: ~/workspace                            │
╰───────────────────────────────────────────────────╯

  Tip: New Build faster with Codex.

Token usage: total=120,608 input=118,938 output=1,670
To continue this session, run codex resume 019e0ee1-a784-7302-b14f-be692712c7c8


› create a python file with logic to support single linked list


⚠ Model metadata for `ollama/devstral:24b` not found. Defaulting to fallback metadata; this can degrade performance and cause issues.

• {"name": "update_plan", "arguments":{"plan":[{"step":"Create Python file for single linked list implementation", "status":"in_progress"}]}}

elasticdotventures · 2 months ago

⚠ Model metadata for ollama/devstral:24b not found. Defaulting to fallback metadata; this can degrade performance and cause issues. • {"name": "update_plan", "arguments":{"plan":[{"step":"Create Python file for single linked list implementation", "status":"in_progress"}]}} ```

Devstral definitely won't work unless it's a variant you trained yourself. Devstral is a generative code 'chat' model, but it predates any type of tool calling. Devstral -- or any chat code generative model might guess a tool call once or twice correctly based on samples but it's going to go off rails quickly after a few sessions, definitely not suitable for a harness like codex. (try aider or something that was created before tool calling)

I'd suggest something like qwen36 served using llamacpp for unsloths gguf or vllm for AWQ depending on the amount of vram you have, you need something that has tool calling it's training corpus, or to FT yourself.

ACodingfreak · 2 months ago

@elasticdotventures - I am not sure how devstral:24b is a reason here. I tried even gemma4 and found similar issue as shown below. Respective tool call has to be executed by the codex but does not seems to do so. I am using litellm as the gateway to access the model which is hosted by ollama.

codex --profile remote
╭─────────────────────────────────────────────────╮
│ >_ OpenAI Codex (v0.130.0)                      │
│                                                 │
│ model:     ollama/gemma4:e4b   /model to change │
│ directory: ~/workspace                          │
╰─────────────────────────────────────────────────╯

  Tip: New Build faster with Codex.


› Create Python file for single linked list implementation


⚠ Model metadata for `ollama/gemma4:e4b` not found. Defaulting to fallback metadata; this can degrade performance and cause issues.

• {
  "name": "exec_command",
  "arguments": {
  "cmd": "touch linked_list.py",
  "justification": "I need to create a new file, linked_list.py, to hold the Python implementation of the single linked list.",
  "sandbox_permissions": "require_escalated"
  }
  }