You shipped 4.2 three weeks ago and your App Store rating slid from 4.6 to 4.3. Somewhere in the reviews is the reason. But your reviews are split across the Apple App Store, Google Play, and G2, nobody reads all three, and the one person who skims them can't tell you whether "sync is broken" is a 4.2 regression or a complaint that's been there since 3.9. Review platforms hand you exactly the metadata that would answer this, the app version attached to each review, and almost every team throws it away.

Can you build a system that doesn't? Yes. This is the conversation-intelligence machine pointed at a different source: a structured extraction pass, clustering with stable theme identity, analyst agents reading summaries, a weekly scheduled run with committed state. That anchor playbook explains the shared machinery in depth, and this post won't repeat it. What's different here, and what this post covers, is the ingestion reality per store, the fact that review volume is low enough to change the extraction economics, and the two analyses reviews uniquely enable: theme-by-version segmentation and star-rating drivers.

What you're building

Every Monday, without you touching anything:

A theme-by-version matrix: complaint clusters as rows, app versions as columns, counts and average star rating in each cell. The money view. "Widget crashes on launch: zero mentions through 4.1, 34 mentions on 4.2" is a sentence that ends a debugging argument.
A star-driver leaderboard: which themes dominate 1-star reviews versus which show up in 5-star reviews, so you know what's dragging the rating versus what's carrying it.
Cross-store synthesis: the same complaint phrased three ways on three platforms gets counted as one theme, not three.
A diff against last week: new themes flagged loudly, known themes get a trend line.

Plus one thing that runs daily, not weekly: a lightweight anomaly check on review velocity. A sudden spike in 1-star reviews is an incident signal, and it should not wait for Monday.

The same pipeline pointed at a competitor's app IDs produces their complaint clusters by version, which is remarkable competitive input. This post mentions that variant and moves on; turning competitor reviews into a sales objection library deserves its own workflow.

The scale math (and why it's friendlier here)

Stated assumptions, check them against your product: a consumer or prosumer app pulling 300-800 reviews per week across all three platforms, averaging 40-80 words each. Call it 500 reviews at roughly 100-150 tokens with metadata. That's 50-75K tokens of new text per week.

Notice what just happened: unlike support chats or agent conversations, which arrive at 20-30 million tokens a week, a week of reviews fits in a context window. This changes the architecture decision the anchor playbook agonized over. You still want per-review structured extraction (clustering needs records, not vibes), but you have two ways to run it:

Inside a normal Claude Code session. The weekly run reads the new reviews in chunks and writes extraction records directly. Viable up to roughly 1,000-2,000 reviews per week; beyond that the session spends more time extracting than analyzing, and quality drifts as context fills.
Through the API via a script, the anchor playbook's pattern. Non-negotiable for the historical backfill: three years of reviews is easily 50,000-100,000 documents, and that one-time job is exactly what the Batch API's roughly 50% discount is for. For the weekly increment at a few hundred reviews, batching is optional; a plain API script or in-session extraction is fine.

The threshold reasoning, not dogma: extraction moves out of the session when volume makes the session about extraction instead of analysis. For most products, that means backfill through the Batch API, weekly increments wherever is simplest for you.

Phase 0: Prerequisites

Claude Code installed and authenticated, Pro/Max/Team/Enterprise for the scheduling and publishing steps.
Store access. For Apple and Google Play, the community AppInsightMCP server covers app market data including reviews. It's a community project, not an official server: review it before trusting it in an automated pipeline, and expect to maintain the integration if the underlying endpoints shift. If you have App Store Connect and Play Console access to your own app, your developer APIs are the sturdier path for your own reviews, and Claude Code can write those export scripts.
A decision about G2 (and Capterra). There is no MCP server we can point to for G2. Your realistic options: check what your vendor account offers for exporting reviews of your own listing, or have Claude Code write a scraper. Know what the second option means: scraping review platforms generally sits somewhere between "against the ToS" and "actively fought," and for competitor listings it's the only option. Get an opinion from whoever owns legal risk before this goes into a scheduled job. The pipeline works fine with G2 as a manual monthly CSV drop.
Python 3.11+ with duckdb, pandas, scikit-learn, and an embeddings option (local sentence-transformers is free and plenty at this volume).
An API key with billing if you're doing the backfill through the Batch API.
A dedicated git repo. Committed state is the continuity mechanism for scheduled runs, same as every playbook in this series.

Phase 1: The repo

review-mining/
├── CLAUDE.md
├── .claude/
│   ├── agents/
│   │   ├── theme-analyst.md
│   │   ├── synthesizer.md
│   │   └── critic.md
│   └── skills/
│       ├── review-mining/SKILL.md    # weekly full run
│       └── review-pulse/SKILL.md     # daily anomaly check
├── scripts/
│   ├── ingest_appstore.py
│   ├── ingest_play.py
│   ├── ingest_g2.py                  # or manual CSV import
│   ├── extract.py                    # backfill + optional weekly path
│   └── cluster_assign.py
├── state/                            # committed
│   ├── clusters.json                 # stable theme IDs, centroids, names
│   ├── versions.json                 # version → release date, per store
│   └── watermark.json                # last review seen, per store
├── data/                             # gitignored: raw reviews, embeddings
└── runs/                             # committed: weekly reports + matrices

Two state files matter beyond what the anchor playbook already covers. versions.json maps release versions to release dates per store, because Apple and Play ship on different days and G2 needs it (next phase). watermark.json is per-store, because the three sources fail independently and a G2 outage shouldn't stall App Store ingestion.

CLAUDE.md carries the standing rules: never renumber theme IDs, never read data/ wholesale into context, every report claim cites a count from a run summary.

Phase 2: Ingestion, store by store

Three sources, three realities:

Apple App Store: reviews carry the app version string in their metadata. Whether you pull via AppInsightMCP or your own App Store Connect script, keep that field; it is the whole point.
Google Play: same story, review metadata includes the reviewer's app version. Play also splits by device and Android version, worth keeping for crash-flavored themes.
G2/Capterra: no app version on the review, just a date. So version tagging here is a join through versions.json: a review dated inside a release window gets tagged with that version, approximately. Mark these records version_source: "inferred" versus "metadata" for the stores. The critic agent will care about this distinction later.

Each ingest script normalizes into one shape: store, review ID, date, star rating, locale, version (plus how the version was determined), title, body. Run all three once, eyeball the output, and only then backfill history.

Phase 3: The extraction schema

Per review, one structured record:

{
  "review_id": "appstore-98211",
  "store": "appstore",
  "version": "4.2.0",
  "version_source": "metadata",
  "stars": 1,
  "locale": "de",
  "themes": ["sync fails after update", "lost data"],
  "sentiment": -0.8,
  "verbatim_quote": "all my projects vanished after the update",
  "quote_is_verbatim": true,
  "schema_version": 2
}

The anchor playbook's extraction rules all apply: cheapest model that passes a 50-review hand-grade, forced verbatim quotes with a flag the critic can verify, versioned schema. Two additions specific to reviews: extraction must handle multiple languages (extract themes in English, keep the quote in the original), and star rating rides along untouched, because it's ground truth chat data never gave you. A conversation makes you infer satisfaction; a review hands you a 1-5 label per document.

Phase 4: Clustering and the version matrix

cluster_assign.py follows the anchor playbook's continuous-clustering design: embed theme phrases, assign to existing clusters within a distance threshold, hold out the rest, promote dense holdout groups to candidate clusters a human confirms. Stable IDs, committed registry, periodic re-cluster with ID remapping. Nothing new to add; read the anchor for the details and the failure modes.

What's new is what you compute from the assignments:

The theme-by-version matrix. For each cluster, review counts and average stars per app version, split by version_source so inferred G2 rows don't masquerade as metadata-grade evidence. A theme whose counts are flat across versions is a chronic gripe. A theme that goes 0, 0, 0, 34 is a regression with a version number attached.
Star drivers. Per cluster, the star distribution. A theme that's 80% 1-star reviews and growing is dragging your rating; a theme concentrated in 5-star reviews is your marketing copy writing itself. Rank clusters by their contribution to low-star volume, not just size.
Velocity baselines. Trailing per-day counts of 1-star reviews per store, persisted into state. This feeds the daily check.

Phase 5: The agents

Same three-role pattern as the anchor: fan-out analysts, a synthesizer, a critic. The analyst here is a theme analyst: it gets one cluster's summary, its version distribution, and 5-8 full example reviews (the only place raw review text enters a context), and answers: what exactly are users describing, which version does the evidence pin it to, does the star distribution say rating-threat or annoyance, and what could not be verified.

The critic's checklist, the part that keeps this report trustworthy:

1. Every quote: grep it verbatim against the raw review files.
   Paraphrased → FAIL.
2. Every version claim ("appeared with 4.2"): recompute the count
   from the matrix, and check version_source. A claim built on
   inferred G2 dates must say so.
3. Every "new theme": confirm the cluster ID is absent from last
   week's snapshot.
4. Counts under 10 reviews: reported as anecdote, not trend.
Return PASS or FAIL with fixes.

Rule 2 is the one this workflow adds. "This complaint shipped with 4.2" is the exact sentence an engineering lead will act on, so it's the sentence the critic re-derives from data before it ships.

The optional response drafting add-on goes here too: the synthesizer drafts replies for the most-visible negative reviews (stores let developers respond; G2 lets vendors respond), and the drafts land in the report as proposals. A human approves and posts. Do not automate the posting; a tone-deaf public reply is worse than silence.

Phase 6: Scheduling: one weekly run, one daily pulse

Two cadences, because the system has two jobs:

/review-mining, weekly. Ingest since watermark, extract, assign clusters, rebuild the matrix, fan out analysts, synthesize with last week's report in hand, critique, commit, publish. Wire it as a cloud routine via /schedule: the routine clones the repo, so it inherits state/ and last week's run automatically. That committed state is the memory between runs; the routine's first step is reading it.
/review-pulse, daily. Deliberately tiny: ingest, count, compare today's 1-star velocity per store against the baseline in state, and post to Slack only on a breach. No agents, no clustering, no report. A 1-star spike usually means an incident, and the signal decays in hours. Daily is a floor, not a ceiling; routines support tighter intervals if a release week warrants it.

Self-hosted alternative for both: system cron running headless mode (claude -p "/review-mining" with --allowedTools scoped down), or a GitHub Actions cron with the official claude-code-action. Either way, add a Stop hook posting run status to Slack, because a review radar that silently died is worse than none.

Phase 7: Sharing with the team

Artifact publish for the weekly report, with the theme-by-version matrix as the centerpiece of the page: an actual table, clusters down, versions across, cells colored by average stars. It's the one view nobody's dashboard has and everyone immediately reads. The usual caveats: publishing works from the CLI and desktop app, not from every automated surface, so a fully autonomous setup may need a human or a different delivery path for this step; org-scoped visibility needs a Team or Enterprise plan.
Slack for the diff: top three movers, any new candidate themes, a link to the page. No official Slack MCP server exists; the community korotovsky/slack-mcp-server is the commonly used one, or a plain webhook script does the job with less surface area.
Route regressions to engineering. A theme the matrix pins to a version is a bug report with volume attached; make the skill file it into your tracker (Linear's official MCP server makes that one line in the skill) rather than hoping someone reads the channel.

Phase 8: Maintenance (what you own now)

Ingestion breakage, three ways. Store endpoints shift, the community MCP server lags behind, and any G2 scraper breaks the moment the page layout changes, which is often. Per-store watermarks plus a validation step ("did every store return a plausible count this week?") turn silent gaps into loud flags. Budget for the G2 path breaking more than the others.
Version-string parsing drift. "4.2.0", "4.2.0 (873)", a hotfix "4.2.1" three days later, stores rolling out the same release days apart. versions.json needs a human touch each release, and the matrix logic needs a decision about whether hotfixes fold into their parent version.
Locale handling. As your app grows internationally, review language mix shifts. Re-grade extractions on non-English samples specifically; that's where quality quietly sags first.
Threshold tuning, twice. The clustering distance threshold (the anchor playbook's standing duty) plus the anomaly threshold on the daily pulse. Too sensitive and the channel learns to ignore it; too dull and it misses the incident it exists for. Log both values into every run snapshot.

What this costs

The cheap end of the text-at-scale family, and worth saying why: cost in these systems tracks token volume through the extraction pass, and reviews are two orders of magnitude smaller than conversations. With the stated assumptions (500 reviews a week, 100-150 tokens each):

Weekly extraction is under a hundred thousand input tokens through a small model. Run in-session, it's a sliver of subscription usage; run through the API, it's a rounding error next to the conversation-intelligence workflow's weekly extraction bill. The Batch API's roughly 50% discount barely matters at this increment, which answers "should I batch": for the weekly run, batch only if it's already how your script works.
The backfill is where the Batch API earns its place: tens of thousands of historical reviews, one time, nothing latency-sensitive. Do it once, with the discount, and never again.
The analysis session is a handful of analysts reading cluster summaries and a few exemplar reviews each, comfortably inside subscription usage. The daily pulse is deliberately near-free: counting is script work, not model work.
The invoice nobody prints: per-store ingestion that breaks independently, a release-by-release versions file someone maintains, two thresholds someone tunes, and a candidate-theme triage duty. Smaller than the conversation-intelligence system's standing load, but it's still a system with an owner, not a cron job.

The four practical challenges

Setup at scale: the friendliest math in the series, but the same architecture. Extraction records, stable clusters, agents on summaries. The scale problem here isn't weekly volume, it's the backfill and the three-store ingestion layer.
Sharing with the team: artifact publish with the version matrix as the centerpiece, Slack for the diff and the daily pulse, and regression-to-ticket routing so version-pinned complaints land on an engineer's queue.
Maintenance: three independently breakable ingestion paths (one of them a scraper living on borrowed time), version-string drift, locale drift, and two thresholds. The G2 path is the part most likely to page you.
Cost: genuinely low in tokens, with the Batch API mattering for the backfill and mostly not for the weekly increment. The real spend is the ingestion upkeep, and no invoice itemizes that.

The managed alternative

App review management is a served market: AppFollow and Appbot both aggregate App Store and Google Play reviews, tag sentiment and topics, and alert on rating drops. If your review problem is mobile-only, they're the fast path. The gaps show at the edges of the category: G2 and web-product reviews sit outside their center of gravity, per-version pain matrices depend on how their tagging maps to your release history, and clustering happens in their taxonomy, not one that emerges from your product's actual complaint patterns.

Doing this with Second Axis

Everything above is the do-it-yourself path. On Second Axis, review mining is a workflow in the marketplace: connect your app store listings and review sources, choose the workflow, and set the goal ("weekly review themes by version, with anomaly alerts on rating drops"). It runs on schedule with the ingestion, clustering, and version matrix handled, and it monitors itself: a store feed that stops returning data or a failed weekly run is found and alerted, so a quiet ingestion break never reads as "no complaints this week."