Page MenuHomePhabricator
Create Task
Maniphest T405597

hCaptcha: Update instrumentation for risk score
Closed, ResolvedPublic
Actions

Description

Summary

For account creations, we store the risk score in serversideaccountcreation. We'll need to update or create a schema for tracking the risk score associated with an edit event, and write code in ConfirmEdit or WikimediaEvents to log the score to the schema.

Acceptance criteria

  • There is a schema for tracking the risk score associated with an edit event
  • The risk score associated with an hCaptcha interaction for edits is saved to an event logging stream

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
EventLogging: Register mediawiki.hcaptcha.risk_score streamoperations/mediawiki-configmaster+12 -0
Refactor CaptchaScoreHooks to use EventSubmittermediawiki/extensions/WikimediaEventswmf/1.46.0-wmf.2+225 -105
Refactor CaptchaScoreHooks to use EventSubmittermediawiki/extensions/WikimediaEventsmaster+225 -105
Instrument hCaptcha risk signal in editsmediawiki/extensions/WikimediaEventswmf/1.46.0-wmf.2+187 -0
Instrument hCaptcha risk signal in editsmediawiki/extensions/WikimediaEventsmaster+187 -0
Customize query in gerrit

Related Objects

Event Timeline

kostajh created this task.Sep 25 2025, 2:17 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 25 2025, 2:17 PM
kostajh edited projects, added Product Safety and Integrity (Sprint Apfel Strudel (Sep 29 - Oct 17)); removed Trust and Safety Product Sprint (Sprint Dadar Gulung (September 8 - September 26)).Sep 29 2025, 7:57 AM
kostajh moved this task from Priority backlog to In refinement on the Product Safety and Integrity (Sprint Apfel Strudel (Sep 29 - Oct 17)) board.Sep 30 2025, 12:01 PM
• sguebo_WMF claimed this task.Oct 3 2025, 11:42 PM
CodeReviewBot added a project: Patch-For-Review.Oct 7 2025, 12:47 AM

sguebo opened https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-secondary/-/merge_requests/85

Draft: WIP - Add hCaptcha/risk_score schema

• sguebo_WMF changed the task status from Open to In Progress.Oct 7 2025, 12:54 AM
• sguebo_WMF moved this task from Backlog to In progress on the WE4.2 Bot detection (WE4.2 hCaptcha editing trial) board.
kostajh moved this task from In refinement to In progress on the Product Safety and Integrity (Sprint Apfel Strudel (Sep 29 - Oct 17)) board.Oct 14 2025, 11:51 AM
kostajh updated the task description. (Show Details)Oct 14 2025, 1:13 PM
• sguebo_WMF updated the task description. (Show Details)Oct 16 2025, 2:50 AM
gerritbot added a comment.Oct 17 2025, 7:31 AM

Change #1196799 had a related patch set uploaded (by Samuel (WMF); author: Samuel (WMF)):

[mediawiki/extensions/WikimediaEvents@master] WIP: Instrument hCaptcha risk signal in edits

https://gerrit.wikimedia.org/r/1196799

OKryva-WMF edited projects, added Product Safety and Integrity (Sprint Mint Choc Chip Ice Cream (Oct 20 - Nov 7)); removed Product Safety and Integrity (Sprint Apfel Strudel (Sep 29 - Oct 17)).Oct 20 2025, 2:29 PM
OKryva-WMF moved this task from Backlog to In progress on the Product Safety and Integrity (Sprint Mint Choc Chip Ice Cream (Oct 20 - Nov 7)) board.Oct 20 2025, 2:29 PM
CodeReviewBot added a comment.Oct 22 2025, 8:54 PM

bearloga updated https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-secondary/-/merge_requests/85

Add hCaptcha/risk_score schema

CodeReviewBot added a comment.Oct 23 2025, 8:48 AM

kharlan merged https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-secondary/-/merge_requests/85

Add hCaptcha/risk_score schema

• sguebo_WMF updated the task description. (Show Details)Nov 6 2025, 7:31 AM
OKryva-WMF edited projects, added Product Safety and Integrity (Crepes au Chocolat (Sprint Nov 10 - Nov 28)); removed Product Safety and Integrity (Sprint Mint Choc Chip Ice Cream (Oct 20 - Nov 7)).Nov 10 2025, 12:15 PM
OKryva-WMF moved this task from Backlog to In progress on the Product Safety and Integrity (Crepes au Chocolat (Sprint Nov 10 - Nov 28)) board.Nov 10 2025, 12:15 PM
gerritbot added a comment.Nov 11 2025, 2:05 PM

Change #1196799 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Instrument hCaptcha risk signal in edits

https://gerrit.wikimedia.org/r/1196799

gerritbot added a comment.Nov 11 2025, 2:23 PM

Change #1203824 had a related patch set uploaded (by Kosta Harlan; author: Samuel (WMF)):

[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.2] Instrument hCaptcha risk signal in edits

https://gerrit.wikimedia.org/r/1203824

gerritbot added a comment.Nov 11 2025, 2:37 PM

Change #1203824 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.2] Instrument hCaptcha risk signal in edits

https://gerrit.wikimedia.org/r/1203824

Stashbot added a comment.Nov 11 2025, 2:38 PM

Mentioned in SAL (#wikimedia-operations) [2025-11-11T14:38:13Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1203824|Instrument hCaptcha risk signal in edits (T405597)]]

Stashbot added a comment.Nov 11 2025, 2:40 PM

Mentioned in SAL (#wikimedia-operations) [2025-11-11T14:40:35Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1203824|Instrument hCaptcha risk signal in edits (T405597)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Stashbot added a comment.Nov 11 2025, 2:47 PM

Mentioned in SAL (#wikimedia-operations) [2025-11-11T14:47:29Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1203824|Instrument hCaptcha risk signal in edits (T405597)]] (duration: 09m 15s)

gerritbot added a comment.Nov 11 2025, 2:53 PM

Change #1203829 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] (WIP) EventLogging: Register mediawiki.hcaptcha.risk_score stream

https://gerrit.wikimedia.org/r/1203829

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.3; 2025-11-19).Nov 11 2025, 3:00 PM
gerritbot added a comment.Nov 11 2025, 3:53 PM

Change #1203842 had a related patch set uploaded (by Samuel (WMF); author: Samuel (WMF)):

[mediawiki/extensions/WikimediaEvents@master] Refactor CaptchaScoreHooks to use EventSubmitter

https://gerrit.wikimedia.org/r/1203842

gerritbot added a comment.Nov 12 2025, 7:35 AM

Change #1203842 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Refactor CaptchaScoreHooks to use EventSubmitter

https://gerrit.wikimedia.org/r/1203842

gerritbot added a comment.Nov 12 2025, 7:37 AM

Change #1204163 had a related patch set uploaded (by Kosta Harlan; author: Samuel (WMF)):

[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.2] Refactor CaptchaScoreHooks to use EventSubmitter

https://gerrit.wikimedia.org/r/1204163

gerritbot added a comment.Nov 12 2025, 9:12 AM

Change #1204163 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.2] Refactor CaptchaScoreHooks to use EventSubmitter

https://gerrit.wikimedia.org/r/1204163

Stashbot added a comment.Nov 12 2025, 9:13 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T09:13:28Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1204088|ext.confirmEdit.hCaptcha: Consider action=submit an edit interface (T409701 T409703 T409415)]], [[gerrit:1204163|Refactor CaptchaScoreHooks to use EventSubmitter (T405597)]]

Stashbot mentioned this in T409415: hCaptcha: Track events for edits in Prometheus.Nov 12 2025, 9:13 AM
Stashbot mentioned this in T409701: hCaptcha: Log challenge event as "saveFailure" in EditAttemptStep.
Stashbot mentioned this in T409703: hCaptcha: Log challenge close and expiry events to VisualEditorFeatureUse.
Stashbot added a comment.Nov 12 2025, 9:15 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T09:15:49Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1204088|ext.confirmEdit.hCaptcha: Consider action=submit an edit interface (T409701 T409703 T409415)]], [[gerrit:1204163|Refactor CaptchaScoreHooks to use EventSubmitter (T405597)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Stashbot added a comment.Nov 12 2025, 9:22 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T09:22:49Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1204088|ext.confirmEdit.hCaptcha: Consider action=submit an edit interface (T409701 T409703 T409415)]], [[gerrit:1204163|Refactor CaptchaScoreHooks to use EventSubmitter (T405597)]] (duration: 09m 21s)

gerritbot added a comment.Nov 12 2025, 9:26 AM

Change #1203829 merged by jenkins-bot:

[operations/mediawiki-config@master] EventLogging: Register mediawiki.hcaptcha.risk_score stream

https://gerrit.wikimedia.org/r/1203829

Stashbot added a comment.Nov 12 2025, 9:27 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T09:27:27Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1203829|EventLogging: Register mediawiki.hcaptcha.risk_score stream (T405597)]]

Stashbot added a comment.Nov 12 2025, 9:30 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T09:29:59Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1203829|EventLogging: Register mediawiki.hcaptcha.risk_score stream (T405597)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Stashbot added a comment.Nov 12 2025, 9:38 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T09:38:09Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1203829|EventLogging: Register mediawiki.hcaptcha.risk_score stream (T405597)]] (duration: 10m 41s)

ReleaseTaggerBot edited projects, added MW-1.46-notes (1.46.0-wmf.2; 2025-11-12); removed MW-1.46-notes (1.46.0-wmf.3; 2025-11-19).Nov 12 2025, 10:01 AM
kostajh moved this task from In progress to Needs QA on the Product Safety and Integrity (Crepes au Chocolat (Sprint Nov 10 - Nov 28)) board.Nov 12 2025, 12:50 PM
dom_walden subscribed.Nov 13 2025, 8:10 AM

@sguebo_WMF @kostajh Where would I see these events? I have tried https://graphite.wikimedia.org/ and https://stream.wikimedia.org/v2/ui/ but cannot find mediawiki.hcaptcha.risk_score.

kostajh added a comment.Nov 13 2025, 10:58 AM
In T405597#11369611, @dom_walden wrote:

@sguebo_WMF @kostajh Where would I see these events? I have tried https://graphite.wikimedia.org/ and https://stream.wikimedia.org/v2/ui/ but cannot find mediawiki.hcaptcha.risk_score.

This is in event logging data, so you can query it with spark3-sql, or on stat1010.eqiad.wmnet, you can also do kafkacat -C -b 'kafka-jumbo1010.eqiad.wmnet:9092' -t codfw.mediawiki.hcaptcha.risk_score -o O | jq to monitor the stream of events.

kostajh moved this task from In progress to QA on the WE4.2 Bot detection (WE4.2 hCaptcha editing trial) board.Nov 13 2025, 10:03 PM
dom_walden moved this task from Needs QA to Done on the Product Safety and Integrity (Crepes au Chocolat (Sprint Nov 10 - Nov 28)) board.Nov 14 2025, 2:47 PM
In T405597#11370208, @kostajh wrote:
In T405597#11369611, @dom_walden wrote:

@sguebo_WMF @kostajh Where would I see these events? I have tried https://graphite.wikimedia.org/ and https://stream.wikimedia.org/v2/ui/ but cannot find mediawiki.hcaptcha.risk_score.

This is in event logging data, so you can query it with spark3-sql, or on stat1010.eqiad.wmnet, you can also do kafkacat -C -b 'kafka-jumbo1010.eqiad.wmnet:9092' -t codfw.mediawiki.hcaptcha.risk_score -o O | jq to monitor the stream of events.

I don't think I have SSH access to production. It doesn't seem worth getting just for this. If someone else can check this, that would be great.

kostajh closed this task as Resolved.Nov 18 2025, 12:20 PM
hector.arroyo moved this task from QA to Done on the WE4.2 Bot detection (WE4.2 hCaptcha editing trial) board.Nov 20 2025, 10:14 AM
kostajh mentioned this in T410992: hCaptcha: Record risk score for failed edits.Nov 25 2025, 9:27 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits