AI code review (Claude, maybe Codex)

Mon Apr 6 10:33:11 PDT 2026

On 6 Apr 2026, at 17:41, Hauke Mehrtens wrote:

> Hi,
>
> I added Claude code review recently to OpenWrt, see:
> https://github.com/openwrt/openwrt/blob/main/.github/workflows/claude-code-review.yml
>
> This GitHub Action will review a PR when an OpenWrt member adds a comment with "/claude". This will trigger a GitHub Action which runs Claude Code to perform the code review.
>
>
> Summary of the current manual test run
> ======================================
> I think this is helpful and it finds real problems. It is a bit nitpicky, but the false positive rate is pretty low.
> It does not only look at the diff itself, but it starts an agent which also looks at the full file being modified. When a new DTS file is added, it often looks at similar DTS files and suggests making the new one look more like the existing ones. It also reads all existing comments on the PR.
>
> The review is pretty expensive; a PR review costs between $0.20 and $1.50, with an average price per review of $0.50. This depends on the number of turns it does. We can restrict the maximum number of turns, but it will reduce the quality of the review.
> I am using the Claude API with their normal API pricing. The review is done using Claude Sonnet 4.6.
>
> The claude-code-action looks like beta software to me. We are using a fork because the normal version does not support reviews of PRs from forked repositories. This was fixed in version 1.0.49, but then reverted due to another problem. It is also not well documented what permissions it needs and how to configure it correctly.
> Upstream is already 12 versions ahead of my two-week-old fork.
> https://github.com/hauke/claude-code-action/tree/v1.0.77-fixed
>
> Here is the full list of PRs where it ran:
> https://github.com/openwrt/openwrt/actions/workflows/claude-code-review.yml?query=is%3Asuccess
>
>
> Current problems
> ================
> We do not have enough reviewers for all the PRs we receive. Many of the PRs adding new devices have the same problems. AI can catch most of them, so we no longer have to handle them manually. I hope AI can help us reduce the load on PR reviews, increase the quality of PRs, and also increase the number of PRs we will merge. If we activate it to run directly after PR creation, contributors will also get fast feedback.
>
> I think AI would have found the recently discovered and fixed security problems in mdnsd and LuCI already when the PR was created.
>
>
> Open Source sponsoring
> ======================
> I have applied for the Open Source maintainers programs from Claude and OpenAI Codex:
> https://developers.openai.com/community/codex-for-oss
> https://claude.com/contact-sales/claude-for-oss
> (The form for Claude does not show for me in Firefox.)
>
> OpenAI Codex is similar to Claude Code as far as I understand.
> The Claude program only offers their Claude Max plan, but this does not work with CI pipelines. We actually need API credits. The OpenAI Codex program looks like it matches our use case.
>
> I also applied for Codex Security, which should be able to find and fix security problems.
>
>
> Future plan
> ===========
> I want to run a good AI agent on all PRs automatically when they are opened and when they are updated. This should apply to all repositories under the OpenWrt GitHub organization. If we were to fully pay for Claude ourselves, this would cost around $200 per month. I hope to get some sponsorship from Anthropic and/or OpenAI to make this happen.
>
> Once we have that, I would first integrate it into our main repository and then into LuCI.
>
> Based on the code review results and what it missed or wrongly suggested we should create documentation the AI can use to improve further reviews. We should add a CLAUDE.md and AGENTS.md or a special version for code reviews which we point the AI to.
>
>
> How do other Open source projects handle this? Do they have a summary of what they do and how well it works for them?
>
> Does someone know other alternatives to Anthropic Claude Code and OpenAI Codex? Does someone know other ways to get sponsorship for this?
>
> Do you agree on running this automatically on all PRs?
>
>
> Feel free to try it yourself by adding a comment with /claude to a PR. It works for all people with commit permission.
>
>
> Example Reviews
> ===============
> Here are some recent examples. The num_turns is the number of turns the AI used for the review. It ran grep to look for similar files, or requested more context, for example. total_cost_usd is the cost of the action in USD, excluding taxes.
>
> Sysupgrade fixes, including related to apk#17847
> https://github.com/openwrt/openwrt/pull/17847#issuecomment-4190389607
>   "num_turns": 11, "total_cost_usd": 0.2514354
>
> generic: net: phy: realtek: add 5G and 10G PHY support#22563
> https://github.com/openwrt/openwrt/pull/22563#issuecomment-4188211274
>   "num_turns": 34, "total_cost_usd": 1.01998215,
>
> Fix setup of non-80MHz 802.11s mesh interfaces on EHT hardware#22644
> https://github.com/openwrt/openwrt/pull/22644#issuecomment-4185594305
>   "num_turns": 14, "total_cost_usd": 0.23388720000000002,
>   It somehow stopped after creating an example patch description which I requested in an other comment.
>
> netifd/odhcp6c: convert DHCP protocol handlers to ucode#22751
> https://github.com/openwrt/openwrt/pull/22751#issuecomment-4184960743
>   "num_turns": 20, "total_cost_usd": 0.4492875,
>
> ramips: mt7621: fix reset hang and re-enable LIST_HARDENED#22724
> https://github.com/openwrt/openwrt/pull/22724#issuecomment-4183374970
>   "num_turns": 14, "total_cost_usd": 0.38940090000000005,
>
> mediatek: filogic: add support for Hilink RM65 development board#22683
> https://github.com/openwrt/openwrt/pull/22683#issuecomment-4164682965
>    "num_turns": 54, "total_cost_usd": 0.6598969499999999,
>
> mediatek: add Huasifei WH3000 Pro NAND support#22694
> https://github.com/openwrt/openwrt/pull/22694#issuecomment-4164295740
>   "num_turns": 26, "total_cost_usd": 0.48784635000000004,
>   It did a incorrect suggestion to change the SPDX-License header
>
> realtek: i2c: rtl9300: backport rtl9607c i2c support and speed patches#22663
> https://github.com/openwrt/openwrt/pull/22663#issuecomment-4164086813
>   "num_turns": 33, "total_cost_usd": 0.6598452,
>
> realtek: dsa: move configuration variables around#22438
> https://github.com/openwrt/openwrt/pull/22438#issuecomment-4164059604
>   "num_turns": 15, "total_cost_usd": 0.41437409999999997,
>    It complained about a change in the PR which was actually not there
>
> Older one:
> realtek: stc8: use flexible array member#22506
> https://github.com/openwrt/openwrt/pull/22506#issuecomment-4114848378
>   "num_turns": 8,  "total_cost_usd": 0.24136304999999997,
>
> Hauke

hi

I'd like to start this out with an immediate acknowledgement of my minimal involvement in owrt dev at this stage, so my opinion here may count for cents. I'd like to follow that up by saying that my involvement has been because of life-limiting factors (a problem which we all know), and I've been managing to reduce those somewhat

that said: this mail/action was an extremely dismaying thing to see

dimensions of why:

 - these platforms are subsidised (in the extreme) by (provably society-damaging) VC-funds; any attempt at building infrastructure upon this without significant review/planning/estimation strikes me as potentially high risk
 - damned near _every_ quantitative study observing reliance upon these systems have found them to be a cognitive net-negative for those relying upon them
 - the (very obvious) aspect of _everything_ about these systems being built on a strip-mined commons (with no intent to contribute back);
   - the intent of this is not to get into litigating the thesis and dimensions of open source, but simply to indicate: _these companies do not give a single fuck_

what is your proposition for how the openwrt project should handle "review feedback" with model-generated code (with the copyright licensing implications therein)?

how do you propose handling the inevitable iteration-loop-increase that the "oh, openwrt reviews PRs with models, it'll be fast now" dynamic is almost certain to result in?

I note in your mail that you indicate that there has been a lack of reviwers, so I want to ask:

 - what type of review is required?
 - what expectations of competency/exposure are wanted?
 - what shape of CTA (call to action) has been attempted to find more/applicable reviewers?

since I am the one sending this mail, I will outright offer myself as tribute: I'm down to take on some more review capacity. I've got a bit of smarts and clue all over, and where I don't know I ask/learn

yours in hope
-J