Issues with the review promp
What feature would you like to see?
I think there are a few very problematic issues with the review prompt. The problem is that the following lines in the review prompts ensures that the review leaves the many serious issues completely alone and does not flag them, leave the codebase in a buggy state, without the user being aware.
"The bug is discrete and actionable (i.e. not a general issue with the codebase or a combination of multiple issues)."
It is problematic that this is not flagged. This makes it ignore very serious issues with the codebase, because they are a combination of multiple factors. I would suggest that this line is either removed or a new review feature is introduced, specifically for reviewing such issues.
"The bug was introduced in the commit (pre-existing bugs should not be flagged)."
This is also very problematic, there are often pre-existing bugs that are in code that has not been either not reviewed before or that wasn't found in an earlier review, not flagging it means it simply stays. I'm guessing the idea behind this is that there may be a bug backlog and you don't want to call out known bugs. I would suggest though, that you instead tell the model to flag the bug as a pre-existing bug, rather than ignoring it, making it up to the user to decide what to do with the bug.
Additional information
_No response_
This issue has 4 comments on GitHub. Read the full discussion on GitHub ↗