Page MenuHomePhabricator
Create Task
Maniphest T173504

Audit usages of integration tests
Closed, DeclinedPublic
Actions

Description

https://gerrit.wikimedia.org/r/#/c/372383/ demonstrates that a single change to Parsoid's output HTML can break dozens of tests in MCS. This feels like it can be avoided.

While an integration test is useful for catching these changes, only one test should have failed in this situation and should have specifically been testing against the HTML output.

I suggest we simplify these tests either by converting them to unit tests or using smaller articles.

We should discard any tests that are not adding any additional value.

Related Objects
Search...

Event Timeline

Jdlrobson created this task.Aug 17 2017, 2:11 PM
• bearND added a comment.Aug 17 2017, 2:18 PM

It's not really dozens of tests. It affected two tests of the same page (one for mobile-sections, another for the formatted endpoint), but they have multiple HTML files associated with them. So, to me the question is: should we keep the HTML files?

Jdlrobson added a comment.Aug 17 2017, 2:21 PM

What purpose do the HTML files serve?

• bearND added a comment.Aug 17 2017, 5:47 PM

I use them for debugging. I don't need them very often, though. So, if we want to get rid of them that wouldn't be a problem for me.

Jdlrobson added a comment.Aug 17 2017, 9:14 PM

Sounds like we should get rid of them? I'm not clear how to use them to debug... not sure if that says something! :)

• Jrbranaa subscribed.Aug 23 2017, 10:55 PM
• bearND added a comment.Aug 24 2017, 2:08 AM

I don't mind removing the HTML files from the repo. I would probably put the writing of the HTML files behind a flag, so it can be used when needed.

• bearND created subtask T175821: Remove HTML files stored in repo for diff tests.Sep 13 2017, 11:27 AM
• bearND moved this task from Needs triage to Backlog on the Product-Infrastructure-Team-Backlog-Deprecated board.
• bearND closed subtask T175821: Remove HTML files stored in repo for diff tests as Resolved.Sep 29 2017, 4:54 PM
• Mholloway closed this task as a duplicate of T151091: MCS uses integration tests which are slow..Sep 21 2018, 3:00 PM
• Mholloway reopened this task as Open.Sep 9 2019, 4:57 PM

This isn't really a duplicate; an audit would be useful in its own right to determine the scope of work needed to eliminate these.

• Mholloway added a project: Page Content Service.Sep 9 2019, 4:58 PM
LGoto closed this task as Declined.Oct 9 2020, 4:50 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits