codex web tries to look at supplied images via code or orc instead visually

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #4819

Powered by Codex Action

(Note: this comment was generated by Codex.)

Thanks for the report! This looks like a duplicate of #4938, which is tracking image handling/interpretation in Codex Web. We’ll track it there. If you think this is an incorrect diagnosis, please reply and we’ll take another look.

Re-opening because codex decided this issues were duplicates of each other...

@FreddyE1982 Could you please explain how you know the images are being supplied via code? From my testing and from the design, it is getting processed to be understood visually by the model

Could you also tell me what the image file type is?

@FreddyE1982 Could you please explain how you know the images are being supplied via code? From my testing and from the design, it is getting processed to be understood visually by the model

@joeytrasatti-openai Its a regular png file and in the protocol codex will say stuff like "I am installing ocr to inspect the image" or some other comment about installing some type of package to inspect the image.

Or is that the model preparing to inspect it visually and I´m just missinterpreting this? In that case...I´m sorry making a fuzz,

No worries, @FreddyE1982 , I did some further testing, the model seems pretty bad at pulling instructions from an image. Looking into this further, but i can confirm it does indeed get fead the image directly. I would advise not using images for instructions, instead using images for design references.

I have the same issue. My use case is that I have a game and a command that can generate a screenshot. All I want is for Codex to be able to inspect the generated png file and test that the change worked correctly. E.g. "verify in the generated screenshot that the button is now red". But it tries to inspect the image programatically / with OCR instead of feeding the image to the model.