Background

The impact dashboard (T430096: Add an impact dashboard for the Starter Kit) reports "tasks completed" based on client-side analytics events:
a task counts as done when the user clicks the completion button in StarterKit, which emits a starterkit.essential.{tool}.complete event and writes a task_completions row.

Problem

The in-app click is a weak proxy for what we actually want to measure — the on-wiki action the task teaches. Auditing the HotCat task (install-gadget) against public wiki data showed both failure directions:

• False negative: a user on kai.wikipedia.org opened the suggested article and added a category with HotCat 49 seconds later (added [[Category:Mbèesu]] using HotCat), but never returned to the app to click "Done". A real completion, counted as nothing.
• False positive: the only recorded complete event was clicked 8 seconds after opening the article, with no corresponding edit anywhere on the wiki.

What data we have

Since 2026-07-18, the open-article CTA event carries username, wiki_url, title, and a timestamp — a complete tuple for later on-wiki verification. HotCat edits are publicly identifiable via their edit summary (... using [[Help:Gadget-HotCat|HotCat]]).

Definition — what does "HotCat task done" mean

Τhe user's own edit gave the suggested article a category, verified server-side. The in-app "Done" click becomes UI-only: today the flow checks the article's category status merely to enable the button, and the click creates the record. Under the new definition the successful check itself records the completion; the click just navigates. The check must therefore also confirm the user made the edit (one revisions?rvuser= call) — "article has a category" alone is too weak to create a record, since a suggested article can be categorized by someone else.

Mechanism proposed

Οne shared verification module, invoked from three trigger points. Because the HotCat task is repeatable (task_completions.run_count increments per run, T431065) and completions must be credited immediately when verifiable (a user must never see a task pending that is actually done), verification is tracked per run in a ledger keyed by article title:

• Verification ledger: a task_verifications table, one row per credited run — wiki_url, task_key, username, article_title, rev_id, verified_at, source, with UNIQUE(wiki_url, task_key, username, article_title). This enforces the rule that re-doing the task on the same title never credits twice, makes concurrent trigger paths race-safe (no double run_count increment), and records which titles each run completed. Crediting = transactional ledger insert + existing run_count upsert; task_completions needs no migration.
• Trigger 1 — in-app check: the existing category-status check on step 3 (visibilitychange / "Check again"), extended to attest the user's edit; on success it credits via the module (idempotent, so repeat checks are no-ops).
• Trigger 2 — completion-status read: whenever the client fetches essential-task completion status (dashboard load, task view load, new tab, refresh — login is just one instance), the server flags pending opens on the session wiki (cheap local lookup, pendingVerification in the response) and the task card shows a loading state only in that case while the module verifies. This covers the user who completes the edit and then opens the dashboard in a fresh tab without ever returning to the task tab. Hard timeout (~5s): on slow/unavailable wiki API the card falls back to current state; the sweep is the backstop. Idempotency makes repeated reads safe; a per-open last_checked_at cooldown caps API calls for never-completing pending opens.
• Trigger 3 — periodic sweep: an in-process timer (not a separate cron process — single SQLite writer, no cross-process locking, credit + ledger insert trivially atomic) verifying pending opens from the last ~30 days. Covers users who never return; the lookback bounds the scan, it is not a correctness rule — no completion windows, no watermarks.
• Pending definition: a titled open-article event whose (user, wiki, task, title) is not in the ledger, within the sweep lookback.
• Historical backfill: a one-off --all run (dry-run + --apply) crediting verifiable completions from existing events. Only titled events are verifiable under this definition; the 5 pre-2026-07-18 title-less opens were each checked manually — exactly 1 is creditable (kai.wikipedia.org, title recovered from the user's public HotCat edit) and can be credited from the recovered data.

Scope — HotCat first, then the same on-wiki-signal review task-by-task for the other essential tasks (each has a different verifiable artifact: created article, imported infobox, …).

Acceptance criteria

• Completion is recorded by the successful server-side verification, not the Done click (click is UI-only)
• Verification attests the user's own edit, not just the article's category status
• Verification triggered on any completion-status read (dashboard/task view/new tab, incl. login), with conditional loading state on the task card and timeout fallback
• task_verifications ledger: per-run crediting keyed by title, race-safe, same title never credits twice
• In-process periodic sweep verifying pending opens (dry-run + --apply for the script entry point)
• Historical backfill run (--all) credits verifiable completions from existing events
• Impact dashboard counts reconciled completions — delivered by T433369: Redefine "task completion" for the Create essential articles task using on-wiki signals and make it repeatable phase 5 (MR !152), which loops VERIFIABLE_TASKS (already contains install-gadget) and ships with the coupled T433369: Redefine "task completion" for the Create essential articles task using on-wiki signals and make it repeatable phases 1+4+5+6 deploy window. Not separate work under this task.