fetch-codex-manual.mjs fails after codex-manual.md redirect drops x-content-sha256

+1

I reproduced this on Codex CLI 0.144.1, macOS 26.5, with the current main helper matching the installed system skill.

One important detail for choosing the fix: preserving x-content-sha256 from the initial 308 response is not valid for the final manual body. In my live reproduction, the redirect supplied:

x-content-sha256: c154a9c6690949e80084031ff962f27eaf0bb6f414eec858672017f4d7fb831d

while the SHA-256 computed from the final redirected manual body was:

4c7550f9e28ae3ce8da2a580be10200cb018a1960cc62f53b25406f3c52cee2e

I validated a client-side fallback with the following behavior:

• A missing hash header is treated as absent rather than malformed.
• A present but invalid hash header remains an error.
• When HEAD supplies a hash, the existing fast cache-hit path remains.
• Otherwise the helper performs GET, computes the body SHA-256 locally, and uses that digest to validate/cache the manual.
• If either HEAD or GET supplies a valid hash, the fetched body must still match it.
• A previously cached body with the same computed digest is reported as current without rewriting it.

Regression checks covered:

1. No hash header: fetch succeeds and subsequent fetch recognizes the cached body.
2. Valid hash header: existing HEAD cache fast path remains.
3. Incorrect hash header: body mismatch is rejected.
4. Live endpoint: first run reported updated; second run reported already current.

A server-side fix that restores the correct hash on the final learn.chatgpt.com response would also preserve the original helper design. Until then, computing the final HTTPS body digest provides a small client-side compatibility fallback without weakening mismatch checks for responses that do provide the header.

Additional reproduction from macOS:

• Installed Codex CLI: 0.143.0.
• The installed helper SHA-256 is f53eb6d2f286e9efcc397e8bee93a938e37296c90953e4e06e94899ef1b6c363, byte-identical to both the rust-v0.144.1 release and current main.
• This environment uses a local HTTP(S) proxy. The proxy preserves x-content-sha256 on the initial 308, while the final 200 from learn.chatgpt.com still lacks the header. This rules out proxy header stripping as the cause in this reproduction.
• The redirect supplied 148b387f4b70f506f21d582ba27d499ba1fbf10908ae89a67cbadf4df64317db, while repeated downloads of the final body produced stable SHA-256 3b1b65de3fdd027d4a034276531f1ea2cc9ded2db4bf4569469977c8875c6065.

This further supports an endpoint/helper contract regression rather than a local configuration problem. We are not applying a local patch because the docs fallback path keeps normal usage unblocked.

I'm also getting integrity check failures every time codex tries to load data for its official manual from online. This is for codex 0.144.1 on macOS. Seems like it'd be good if there was integration tests that that automatically compared the checksums in codex vs what is requested from online.

Confirming this also affects my Codex Desktop environment:

• Bundled Codex CLI: 0.144.0-alpha.4
• macOS 15.7.3 (24G419)
• curl 8.7.1
• Helper SHA-256: f53eb6d2f286e9efcc397e8bee93a938e37296c90953e4e06e94899ef1b6c363

Sandboxed networking is enabled and both developers.openai.com and learn.chatgpt.com are explicitly allowlisted and reachable. After restarting Codex, the helper still fails consistently with Manual response is missing x-content-sha256.

The initial 308 contains the header, while the final redirected 200 does not. The official Docs MCP fallback works, but this causes repeated visible failure/fallback noise across tasks. This appears to be the same endpoint/helper contract regression rather than a local networking or allowlist problem.

Additional Windows/CLI reproduction and tested client-side compatibility patch

I reproduced this on Codex CLI 0.144.3 / Windows / PowerShell with the installed helper byte-equivalent to the current official source blob b2605520dc54aa3b71769e90887b9fdf06c97da4 before modification.

The observed response chain was:

developers.openai.com/codex/codex-manual.md
  -> 308, x-content-sha256 present
  -> https://learn.chatgpt.com/docs/codex-manual.md
  -> 200, x-content-sha256 absent

The proxy preserved the initial 308 checksum header, so proxy header stripping was not the root cause in this reproduction. The final 200 simply did not include the header.

It is also unsafe to reuse the checksum from the 308 for the final body. In this run:

308 x-content-sha256:
6c175fbbc12aada07a1a3a3d25b3e2705d1d5cb8bfc0ae65ddf4a90d4ffba3cc

Final body SHA-256:
8cb97555145b766f396b9318bf3bcbcc81bed91fd716a4f7a6ebf1151fefc1d9

Final body size:
764023 bytes

I applied and tested the following compatibility semantics locally:

1. Treat a missing checksum header as absent, while continuing to reject a present but malformed header.
2. Preserve the existing header-backed HEAD-only cache fast path when a valid checksum is available.
3. If the checksum is absent, perform GET and compute SHA-256 from the final body.
4. If either HEAD or GET provides a valid checksum, require the body digest to match it.
5. In the headerless path, compare the locally computed digest with the existing cached body and report hit without rewriting an unchanged cache.
6. Expose verificationMode as header-sha256 or local-sha256 so the fallback is explicit and not represented as a server-provided checksum.

Core diff against the current helper:

@@
 const readHeaderSha = (response) => {
   const value = response.headers.get(HASH_HEADER);
-  if (!value || !/^[a-f0-9]{64}$/i.test(value)) {
-    throw new ManualFetchError(`Manual response is missing ${HASH_HEADER}.`);
+  if (!value) return null;
+  if (!/^[a-f0-9]{64}$/i.test(value)) {
+    throw new ManualFetchError(`Manual response returned an invalid ${HASH_HEADER}.`);
   }
   return value.toLowerCase();
 };
@@
-  const expectedSha256 = readHeaderSha(headResponse);
+  const headSha256 = readHeaderSha(headResponse);
@@
-  const cachedManual = await readCachedManual(resolvedCacheDir, expectedSha256);
-  if (cachedManual !== null) {
+  if (headSha256) {
+    const cachedManual = await readCachedManual(resolvedCacheDir, headSha256);
+    if (cachedManual !== null) {
       const outline = buildOutline(cachedManual);
       const outlineText = outlineMarkdown(outline);
       await writeOutline(resolvedCacheDir, outlineText);
 
       return {
         outlineText,
         status: {
           manualUrl,
-          headerSha256: expectedSha256,
-          fetchedManualSha256: expectedSha256,
+          verificationMode: "header-sha256",
+          headerSha256: headSha256,
+          fetchedManualSha256: headSha256,
           manualHashMatches: true,
           cacheStatus: "hit",
           cacheDir: resolvedCacheDir,
           manualPath,
           outlinePath,
           checkedAt,
           lineCount: outline.lineCount,
           headingCount: outline.headingCount,
         },
       };
+    }
   }
@@
   const getHeaderSha256 = readHeaderSha(getResponse);
-  if (getHeaderSha256 !== expectedSha256) {
+  if (headSha256 && getHeaderSha256 && getHeaderSha256 !== headSha256) {
     throw new ManualFetchError(
       `${HASH_HEADER} changed between HEAD and GET for ${manualUrl}.`
     );
   }
 
   const manualText = await getResponse.text();
   const actualSha256 = sha256(manualText);
-  const manualHashMatches = actualSha256 === expectedSha256;
+  const expectedSha256 = headSha256 ?? getHeaderSha256;
+  const manualHashMatches = expectedSha256 ? actualSha256 === expectedSha256 : true;
   if (!manualHashMatches) {
     throw new ManualFetchError(
       `${HASH_HEADER} did not match the fetched manual body for ${manualUrl}.`
     );
   }
 
-  await writeCachedManual(resolvedCacheDir, manualText);
-  const outline = buildOutline(manualText);
+  const verificationMode = expectedSha256 ? "header-sha256" : "local-sha256";
+  const cachedManual = await readCachedManual(resolvedCacheDir, actualSha256);
+  const cacheStatus = cachedManual !== null ? "hit" : "updated";
+
+  if (cacheStatus === "updated") {
+    await writeCachedManual(resolvedCacheDir, manualText);
+  }
+  const outline = buildOutline(cachedManual ?? manualText);
   const outlineText = outlineMarkdown(outline);
   await writeOutline(resolvedCacheDir, outlineText);
@@
     status: {
       manualUrl,
+      verificationMode,
       headerSha256: expectedSha256,
       fetchedManualSha256: actualSha256,
       manualHashMatches,
-      cacheStatus: "updated",
+      cacheStatus,

Regression tests covered:

PASS missing checksum headers use local sha256 and preserve cache hits
PASS valid checksum header permits a HEAD-only cache hit
PASS a supplied checksum that does not match the body still fails
PASS --help documents the interface without fetching

tests: 4
pass: 4
fail: 0

Live endpoint verification after applying the patch:

{
  "verificationMode": "local-sha256",
  "headerSha256": null,
  "fetchedManualSha256": "8cb97555145b766f396b9318bf3bcbcc81bed91fd716a4f7a6ebf1151fefc1d9",
  "manualHashMatches": true,
  "cacheStatus": "updated",
  "lineCount": 15846,
  "headingCount": 121
}

A second live run produced the same digest with:

{
  "verificationMode": "local-sha256",
  "manualHashMatches": true,
  "cacheStatus": "hit"
}

Security boundary: local-sha256 provides download/cache consistency over the final HTTPS body, but it is not an independently server-authenticated digest. A server-side fix that restores a correct x-content-sha256 or standard Content-Digest on the final 200 remains preferable. The fallback is explicit, preserves all mismatch checks when a header is provided, and does not trust the mismatching redirect checksum.

I subsequently restored my local installed helper to the exact official 0.144.3/main bytes while waiting for an upstream decision; the results above are from the tested first-round compatibility patch, not a claim that this behavior is currently shipped by OpenAI.

Rather than fixing only the current redirect, I suggest treating this as a resilience contract for the bundled docs skill.

Proposed behavior

1. Do not require a successful HEAD request. A CDN may omit or change headers between HEAD, redirects, and the final GET. Use GET as the source of truth; use ETag / Last-Modified for conditional requests when available.
2. Prefer an authenticated response digest when supplied. Accept a valid standard Content-Digest or the existing x-content-sha256; a malformed digest or body mismatch remains a hard validation failure.
3. Degrade explicitly when no server digest exists. Hash the final HTTPS body locally for cache/change detection and report verificationMode: "local-sha256" (not as equivalent to a server-authenticated digest).
4. Never discard a last-known-good cache on refresh failure. Return the cached manual with a structured stale warning when the endpoint is unreachable or response validation fails.
5. Fall back cleanly at the skill layer. If neither a valid response nor a cache is available, route to the official Docs MCP path and report a concise warning instead of surfacing the helper stack trace as the primary result.

Suggested acceptance tests

• redirect where the final response omits the custom digest;
• HEAD unsupported or inconsistent with GET;
• 304 Not Modified via ETag / Last-Modified;
• present digest that does not match the body (must fail validation);
• network/5xx failure with a valid cache (must preserve and return stale cache);
• no cache plus fetch failure (must produce a structured fallback signal).

This keeps strict verification whenever the server provides integrity metadata while avoiding a single non-standard header becoming a hard availability dependency for built-in Codex documentation.