Add Comment Moderation experiment

[Jameswlepage]

What?

Adds comment toxicity and sentiment analysis and uses that to moderate comments.

Why?

There are multiple approaches to moderating comments and using AI to do that is a natural fit. For sites that get lots of comments, having a first-line moderation strategy is key to keep up with them.

How?

• Adds a new Comment Analysis Ability that will analyze a comment for toxicity and sentiment
  • Toxicity scoring: Analyzes comments for harmful content and scores Low/Medium/High
  • Sentiment analysis: Detects positive, negative, or neutral sentiment
• Add a new Comment Moderation Experiment that uses this Ability
• When a new comment is left, automatically pass the content of the comment into our analysis Ability. If the sentiment is negative and the toxicity score is above 0.7, put the comment into moderation
• On the comment list screen in the admin, add new columns that output the comment toxicity and sentiment values
• On this same screen, add a new bulk dropdown option to analyze one or more comments. This won't moderate, will just run and save the analysis
  • Lazy analysis: Automatically processes pending comments when the Comments page loads
• Add all tests and documentation

Use of AI Tools

Can't speak to the original code in this PR but in the cleanup, used Claude running Opus 4.6 and also Cursor running Codex 5.3 to do code reviews, some cleanup, tests and documentation

Testing Instructions

• Checkout this PR and run npm i && npm run build
• Enable the Comment Moderation experiment under Settings → AI
• Ensure you have a valid AI connector in place
• Create a new post and add a positive comment to that post
• Visit Comments in the admin and verify badges appear
• Verify there's a bulk action "Analyze with AI" and use that
• Add a new comment with some negative language (I hate this post). Ensure this comment goes to moderation

Screenshots

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: Jameswlepage <isotropic@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: sethrubenstein <smrubenstein@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[codecov]

Codecov Report

❌ Patch coverage is 43.20000% with 213 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.23%. Comparing base (f48f8c9) to head (df9f427).

Files with missing lines	Patch %	Lines
...eriments/Comment_Moderation/Comment_Moderation.php	35.90%	166 Missing ⚠️
.../Abilities/Comment_Moderation/Comment_Analysis.php	59.48%	47 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #155      +/-   ##
=============================================
- Coverage      69.17%   67.23%   -1.94%     
- Complexity       981     1054      +73     
=============================================
  Files             63       65       +2     
  Lines           4649     5024     +375     
=============================================
+ Hits            3216     3378     +162     
- Misses          1433     1646     +213     

Flag	Coverage Δ
unit	67.23% <43.20%> (-1.94%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This PR adds a comprehensive AI-powered comment moderation experiment to WordPress, enabling automated toxicity detection, sentiment analysis, and AI-generated reply suggestions directly within the Comments admin screen.

Key Changes

• Introduces two new WordPress Abilities: ai/comment-analysis for toxicity/sentiment scoring and ai/reply-suggestion for generating contextual reply drafts
• Implements React-based UI with lazy analysis of pending comments and an interactive reply modal with tone selection
• Adds a shared run-ability.ts utility that provides Abilities API client support with REST API fallback

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
webpack.config.js	Adds webpack entry point for comment-moderation experiment bundle
src/utils/run-ability.ts	New shared utility for executing abilities with client/REST fallback
src/experiments/comment-moderation/index.tsx	React entry point mounting lazy analysis and reply modal controllers
src/experiments/comment-moderation/components/ReplyModalController.tsx	Manages reply modal state and WordPress inline reply form integration
src/experiments/comment-moderation/components/ReplyModal.tsx	UI for displaying and selecting AI-generated reply suggestions
src/experiments/comment-moderation/components/LazyAnalysisController.tsx	Detects and processes pending comment analysis on page load
includes/Experiments/Comment_Moderation/Comment_Moderation.php	Main experiment class with column rendering, bulk actions, and asset enqueuing
includes/Experiment_Loader.php	Registers Comment_Moderation experiment in loader
includes/Abstracts/Abstract_Ability.php	Adds get_model_preferences() helper method to base ability class
includes/Abilities/Comment_Moderation/Reply_Suggestion.php	Ability implementation for generating reply suggestions with tone control
includes/Abilities/Comment_Moderation/Comment_Analysis.php	Ability implementation for analyzing comment toxicity and sentiment
includes/Abilities/Comment_Moderation/system-instruction.php	AI system prompt for comment analysis with JSON output format
includes/Abilities/Comment_Moderation/reply-system-instruction.php	AI system prompt for generating contextual comment replies
docs/experiments/comment-moderation.md	Documentation covering hooks, data flow, and testing procedures

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jeffpaul]

Note we'll want to update the screenshot numbers when this gets merged in, I left it with a hash instead of a number depending on when this versus other experiments get merged in.

[jeffpaul]

@Jameswlepage can you add tests here like was done in https://github.com/WordPress/ai/pull/147/changes#diff-0cf348eda48cae411ad9d44a2e5b449603a971f2313b0a849893ff4cf8280c46?

[jeffpaul]

Testing showing all comments when run thru bulk action are getting Failed response:

[dkotter]

@jeffpaul This is ready for review now. I've read through all the comments on the PR and I think I've addressed all actionable items. There is some good back and forth with you and Seth that I'd suggest we pull that out into a new issue just so we don't lose track of it, as I think there's some valuable things we can look at as enhancements to this or even entirely new experiments.

[jeffpaul]

1. Update the bulk action to Analyze Sentiment and Toxicity (while I agree Analyze with AI is MUCH shorter I'm trying to get away from using AI in the editor and admin and more describe things as "this is what happens when you click this".

2. Is there sufficient hooks in place such that the "comment value" / Relevance concept that @sethrubenstein mentions in #155 (comment) and #155 (comment) could reasonably be extended from this work (and similarly have low value comments based on their algorithm/approach go to moderation)?

[dkotter]

1. Update the bulk action to Analyze Sentiment and Toxicity (while I agree Analyze with AI is MUCH shorter I'm trying to get away from using AI in the editor and admin and more describe things as "this is what happens when you click this".

This has been updated now

2. Is there sufficient hooks in place such that the "comment value" / Relevance concept that @sethrubenstein mentions in #155 (comment) and #155 (comment) could reasonably be extended from this work (and similarly have low value comments based on their algorithm/approach go to moderation)?

So yes and no. There's a filter around the system instructions and another around the response schema so in theory, you could use both of those to change the response the LLM gives. There's also then Core filters on the comment list screen that you could use to add a new column for "Value/Relevance".

But the manual processing of comments is done via custom javascript which doesn't have hooks in place so you'd have to add some of your own JS to replicate the full experience.

That said, I'd love to add a comment value / relevance experiment to this plugin though I think that's fine as a separate PR (also some discussion on if that should be part of comment moderation or a separate experiment).