Add a link: instance_occurrence should be based on plaintext, not wikitext
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Tgr
	Jan 9 2021, 3:30 AM

Description

instance_occurrence counts how many times the link text occurs before the point where it should be linked. This is now based on wikitext, which cannot be used for Parsoid HTML. It should be based on plaintext instead.

In other words, when the service suggests changing Scary careening  car to Scary careening  [[car]], currently instance_occurrence is (assuming 1-based indexing) 4 but it should be 3 as we want to disregard the comment. Same goes for template parameters etc.

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Make instance_occurence plaintext-based	research/mwaddlink	main	+116 -114

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	• MMiller_WMF	T252822 [EPIC] Growth: "add a link" structured task 1.0
Resolved	kostajh	T266437 Add a link engineering: backend product specifications
Resolved	kostajh	T261396 Add a link: engineering tasks for initial release
Resolved	Tgr	T271604 Add a link: instance_occurrence should be based on plaintext, not wikitext

Event Timeline

Tgr created this task.Jan 9 2021, 3:30 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 9 2021, 3:30 AM

Tgr added a parent task: T261396: Add a link: engineering tasks for initial release.Jan 9 2021, 3:31 AM

Tgr moved this task from Incoming to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.

Tgr mentioned this in T271124: Add a link engineering: deal with word boundaries (\b) being broken in JS.Jan 9 2021, 3:46 AM

Tgr claimed this task.Jan 11 2021, 10:00 AM

Tgr moved this task from Ready for Development to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.

Change 655547 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[research/mwaddlink@main] Make instance_occurence plaintext-based

https://gerrit.wikimedia.org/r/655547

gerritbot added a project: Patch-For-Review.Jan 12 2021, 6:09 AM

Patch is up; I have no idea how to test it though.
It takes the somewhat lazy approach of only considering top-level text nodes (ie. it ignores anything inside a comment or a template, but also the contents of things like list items or italic text), which matches how mwaddlink selects link anchors in the first place. That means there is no need for recursion, and the behavior should be easy to match in Parsoid HTML (only look at text DOM nodes which are not inside any [typeof] tag), apart from dealing with DisplaySpace and other whitespace differences but that's an unavoidable problem.

Change 655547 merged by jenkins-bot:
[research/mwaddlink@main] Make instance_occurence plaintext-based

https://gerrit.wikimedia.org/r/655547

Maintenance_bot removed a project: Patch-For-Review.Jan 15 2021, 9:10 AM