Run comparison of html extracts again
Closed, ResolvedPublic
Actions

Description

I ran the comparison script, which compare the exract_html fields, again.
Here are the English results:

For the new run I added a 'b' to the file names. The en.v2.txt is from the old run we did a while ago.
(http://jdlrobson.com/summaries/en.2b.html)

So far I've noticed a bunch more issue classes:

showing coordinates in Qatar and United States (probably the order of operations is to blame)
Escaped HTML causes issues in Transformers:_The_Last_Knight: <i id=\"mwCQ\">Transformers</i> and & inside Ariana Grande. See also Logan (Film), DJ Khaled, Keanu Reeves, Beyoncé, Clint Eastwood, Emma Watson and Chris Pine,
undefined in Donald Trump, Barack Obama and Botulism

Details

Subject	Repo	Branch	Lines +/-
Summary: remove nested spans	mediawiki/services/mobileapps	master	+13 -4
Summaries should not contain coordinates	mediawiki/services/mobileapps	master	+13 -0
Do not completely flatten links with child nodes	mediawiki/services/mobileapps	master	+14 -2

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T169242 Develop Page Content Service for Reading Clients
Resolved	None	T177425 Develop General Layer of PCS
Resolved	• Jhernandez	T177426 Develop structured JSON APIs for general consumption
Resolved	• Mholloway	T177431 Develop a Summary JSON API
Resolved	Dereckson	T68374 Enable Hovercards on se.wikimedia.org (Swedish chapter wiki)
Resolved	Jdlrobson	T70860 [GOAL] Graduate Page Previews feature (Popups extension) out of Beta Feature
Resolved	ovasileva	T154635 [EPIC] Deploy page previews to English and German Wikipedia
Resolved	ovasileva	T192622 [EPIC] Page previews post-deploy cleanup
Resolved	Jdlrobson	T173952 Remove A/B testing instrumentation code
Duplicate	None	T167433 Switch all projects to the new (and yet to be built) summary-html endpoint for page previews
Duplicate	None	T167429 Make enwiki and dewiki fetch previews from the summary-html RESTBase endpoint
Resolved	ovasileva	T165018 Page previews can consume new summary-HTML endpoint
Declined	Jdlrobson	T111329 [GOAL] Page previews on mobileweb
Resolved	Jdlrobson	T164010 [EPIC] Strengthen the APIs we provide in reading web maintained extensions
Resolved	ovasileva	T113094 [EPIC] The Page Summary API needs to provide useful content for the majority of articles
Resolved	Jdlrobson	T185050 Run comparison of html extracts again

Event Timeline

• bearND triaged this task as High priority.Jan 16 2018, 11:17 PM

• bearND created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 16 2018, 11:17 PM

Jdlrobson updated the task description. (Show Details)Jan 16 2018, 11:20 PM

Jdlrobson updated the task description. (Show Details)Jan 16 2018, 11:24 PM

Change 404603 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/services/mobileapps@master] Summaries should not contain coordinates

https://gerrit.wikimedia.org/r/404603

gerritbot added a project: Patch-For-Review.Jan 16 2018, 11:38 PM

Change 404609 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/services/mobileapps@master] Do not completely flatten links with child nodes

https://gerrit.wikimedia.org/r/404609

Change 404610 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/services/mobileapps@master] Test case: Nested spans should be removed...

https://gerrit.wikimedia.org/r/404610

After the above 2 patches I re-ran the script: http://jdlrobson.com/summaries/en.2b.jon.html
Those first 2 fixes eradicate all the problems.
The 3rd patch adds a test which captures the Ariana Grande empty parenthetical issue, but this is probably lower importance.

Change 404609 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Do not completely flatten links with child nodes

https://gerrit.wikimedia.org/r/404609

Change 404603 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Summaries should not contain coordinates

https://gerrit.wikimedia.org/r/404603

@bearND I think all but the space in Gal Gadot are fixed now?
I should note I'm aware of this happening, but consider it pretty low priority and minor. In interest of deploying sooner, I'd be keen to open a new task for that and deal with that post-deploy rather than add to the list of changes we're making. What do you think?

@Jdlrobson I think we can deploy to MCS what we have merged so far tomorrow. The switchover has to happen in RESTBase, which should happen after the DevSummit week anyways. There are a few more tasks in MCS outstanding, too. I believe there is a good chance we will have the extra space eradicated as well by then, too. But I'm ok if that happens after switchover, too. Same for the nested <span></span>.

Is http://jdlrobson.com/summaries/en.2b.jon.html is still the latest and the same as what's on master? I haven't run it again but could do so if you haven't.

I reran. http://jdlrobson.com/summaries/en.011718.html is the latest version.

Being bold. I've opened up T185161 for the remaining issue...

Jdlrobson updated the task description. (Show Details)Jan 17 2018, 11:30 PM

Mentioned in SAL (#wikimedia-operations) [2018-01-18T18:40:32Z] <bsitzmann@tin> Started deploy [mobileapps/deploy@669fb5b]: Update mobileapps to 2690899 (T184328 T184557 T177007 T184669 T177430 T185050)

Mentioned in SAL (#wikimedia-operations) [2018-01-18T18:47:35Z] <bsitzmann@tin> Finished deploy [mobileapps/deploy@669fb5b]: Update mobileapps to 2690899 (T184328 T184557 T177007 T184669 T177430 T185050) (duration: 07m 03s)