Page MenuHomePhabricator
Create Task
Maniphest T316849

Audit tests for Druid-based endpoints
Closed, ResolvedPublic
Actions

Description

The various production Druid-based AQS endpoints (collectively known as the Wikistats 2 API) currently have their own special testing fixtures. These fixtures are essentially a "fake" Druid against which to execute tests.

In AQS 2.0, we intend to use a Docker-based actual Druid testing environment, per T311190: Establish testing procedure for Druid-based endpoints. So the details of the AQS 2.0 test implementation will be very different. However, to maintain testing parity with the existing production service, we should audit the existing production tests and ensure that AQS 2.0 has at least that same level of testing coverage before release.

The following is a list of every individual test found in the existing production fixtures.js (the "fake druid" tests). The list is grouped by the fixture name, both to make it more convenient to find the test implementation in fixtures.js if desired, and also because it will probably make sense to mirror those groupings in AQS 2.0. Each test also includes that path being tested, for reference. We will almost certainly use different test data in AQS 2.0, so the date-based parameters are likely to change in the new tests.

newPagesFixtures

  • 'return 400 for new-pages with typo in date' : '/edited-pages/new/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010a00'
  • 'return 400 for new-pages with invalid date (end before start)' : '/edited-pages/new/all-projects/all-editor-types/all-page-types/daily/2017010200/2017010100'
  • 'return 400 for new-pages with invalid granularity' : '/edited-pages/new/all-projects/all-editor-types/all-page-types/wrong/2017010100/2017010200'
  • 'return 400 for new-pages with invalid page-type' : '/edited-pages/new/all-projects/all-editor-types/wrong/daily/2017010100/2017010200'
  • 'return 400 for new-pages with invalid editor-type' : '/edited-pages/new/all-projects/wrong/all-page-types/daily/2017010100/2017010200'
  • 'return 400 for new-pages with invalid project' : '/edited-pages/new/bizarre|project$name/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 400 for monthly new-pages and no full-month dates' : '/edited-pages/new/all-projects/all-editor-types/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for new-pages daily without filters' : '/edited-pages/new/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 and results for new-pages monthly without filters nor hours' : '/edited-pages/new/all-projects/all-editor-types/all-page-types/monthly/20170101/20170210'
  • 'return 200 with results for new-pages with project-family filter' : '/edited-pages/new/all-wikipedia-projects/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for new-pages with uppercase project filter' : '/edited-pages/new/EN.wikipedia.org/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for new-pages with project and editor-type filters' : '/edited-pages/new/EN.wikipedia.org/user/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for new-pages with project, editor-type and page-type filters' : '/edited-pages/new/en.wikipedia/anonymous/non-content/daily/2017010100/2017010200'

editedPagesFixtures

  • 'return 400 for edited-pages with all-projects filter' : '/edited-pages/aggregate/all-projects/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/201701000'
  • 'return 400 for edited-pages with project-family filter' : '/edited-pages/aggregate/all-wikivoyage-projects/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/201701000'
  • 'return 400 for edited-pages with typo in date' : '/edited-pages/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/2017010a00'
  • 'return 400 for edited-pages with invalid date (end before start)' : '/edited-pages/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/daily/2017010200/2017010100'
  • 'return 400 for edited-pages with invalid granularity' : '/edited-pages/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/wrong/2017010100/2017010200'
  • 'return 400 for edited-pages with invalid activity-level' : '/edited-pages/aggregate/en.wikipedia/all-editor-types/all-page-types/wrong/daily/2017010100/2017010200'
  • 'return 400 for edited-pages with invalid page-type' : '/edited-pages/aggregate/en.wikipedia/all-editor-types/wrong/all-activity-levels/daily/2017010100/2017010200'
  • 'return 400 for edited-pages with invalid user-type' : '/edited-pages/aggregate/en.wikipedia/wrong/all-page-types/all-activity-levels/daily/2017010100/2017010200'
  • 'return 400 for monthly edited-pages and no full-month dates' : '/edited-pages/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/monthly/2017010100/2017010200'
  • 'return 200 and results for edited-pages daily with uppercase and .org project filter' : '/edited-pages/aggregate/EN.wikipedia.org/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/2017010200'
  • 'return 200 and results for edited-pages monthly with only project filter but no hour' : '/edited-pages/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/monthly/20170101/20170210'
  • 'return 200 with results for edited-pages with project and editor-type filter' : '/edited-pages/aggregate/en.wikipedia/anonymous/all-page-types/all-activity-levels/daily/2017010100/2017010200'
  • 'return 200 with results for edited-pages with project, editor-type and page-type filter' : '/edited-pages/aggregate/en.wikipedia/user/non-content/all-activity-levels/daily/2017010100/2017010200'
  • 'return 200 with results for edited-pages with project, editor-type, page-type and activity-level filter' : '/edited-pages/aggregate/en.wikipedia/group-bot/content/25..99-edits/daily/2017010100/2017010200'

newRegisteredUsersFixtures

  • 'return 400 for new registered users with typo in date' : '/registered-users/new/all-projects/daily/2017010100/2017010a00'
  • 'return 400 for new registered users with invalid date (end before start)' : '/registered-users/new/all-projects/daily/2017010200/2017010100'
  • 'return 400 for new registered users with invalid granularity' : '/registered-users/new/all-projects/wrong/2017010100/2017010200'
  • 'return 400 for monthly new registered users and no full-month dates' : '/registered-users/new/all-projects/monthly/2017010100/2017010200'
  • 'return 200 and results for new registered users daily without filters' : '/registered-users/new/all-projects/daily/2017010100/2017010200'
  • 'return 200 and results for new registered users monthly without filters nor hours' : '/registered-users/new/all-projects/monthly/20170101/20170210'
  • 'return 200 with results for new registered users with project-family filter' : '/registered-users/new/all-wiktionary-projects/daily/2017010100/2017010200'
  • 'return 200 with results for new registered users with uppercase project filter' : '/registered-users/new/EN.wikipedia.org/daily/2017010100/2017010200'

editorsFixtures

  • 'return 400 for editors with all-projects filter' : '/editors/aggregate/all-projects/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/201701000'
  • 'return 400 for editors with project-family filter' : '/editors/aggregate/all-wikisource-projects/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/201701000'
  • 'return 400 for editors with typo in date' : '/editors/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/2017010a00'
  • 'return 400 for editors with invalid date (end before start)' : '/editors/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/daily/2017010200/2017010100'
  • 'return 400 for editors with invalid granularity' : '/editors/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/wrong/2017010100/2017010200'
  • 'return 400 for editors with invalid activity-level' : '/editors/aggregate/en.wikipedia/all-editor-types/all-page-types/wrong/daily/2017010100/2017010200'
  • 'return 400 for editors with invalid page-type' : '/editors/aggregate/en.wikipedia/all-editor-types/wrong/all-activity-levels/daily/2017010100/2017010200'
  • 'return 400 for editors with invalid user-type' : '/editors/aggregate/en.wikipedia/wrong/all-page-types/all-activity-levels/daily/2017010100/2017010200'
  • 'return 400 for monthly editors and no full-month dates' : '/editors/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/monthly/2017010100/2017010200'
  • 'return 200 and results for editors daily with uppercase and .org project filter' : '/editors/aggregate/EN.wikipedia.org/all-editor-types/all-page-types/all-activity-levels/daily/2017010100/2017010200'
  • 'return 200 and results for editors monthly with project filter but no nor hour' : '/editors/aggregate/en.wikipedia/all-editor-types/all-page-types/all-activity-levels/monthly/20170101/20170210'
  • 'return 200 with results for editors with project and editor-type filter' : '/editors/aggregate/en.wikipedia/anonymous/all-page-types/all-activity-levels/daily/2017010100/2017010200'
  • 'return 200 with results for editors with project, editor-type and page-type filter' : '/editors/aggregate/en.wikipedia/user/non-content/all-activity-levels/daily/2017010100/2017010200'
  • 'return 200 with results for editors with project, editor-type, page-type and activity-level filter' : '/editors/aggregate/en.wikipedia/group-bot/content/5..24-edits/daily/2017010100/2017010200'

editsFixtures

  • 'return 400 for edits with typo in date' : '/edits/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010a00'
  • 'return 400 for edits with invalid date (end before start)' : '/edits/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010200/2017010100'
  • 'return 400 for edits with invalid granularity' : '/edits/aggregate/all-projects/all-editor-types/all-page-types/wrong/2017010100/2017010200'
  • 'return 400 for edits with invalid page-type' : '/edits/aggregate/all-projects/all-editor-types/wrong/daily/2017010100/2017010200'
  • 'return 400 for edits with invalid user-type' : '/edits/aggregate/all-projects/wrong/all-page-types/daily/2017010100/2017010200'
  • 'return 400 for monthly edits and no full-month dates' : '/edits/aggregate/all-projects/all-editor-types/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for edits daily without filters' : '/edits/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 and results for edits monthly without filters nor hours' : '/edits/aggregate/all-projects/all-editor-types/all-page-types/monthly/20170101/20170210'
  • 'return 200 with results for edits with project-family filter' : '/edits/aggregate/all-wikiversity-projects/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for edits with uppercase project filter' : '/edits/aggregate/EN.wikipedia.org/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for edits with project and user-type filter' : '/edits/aggregate/en.wikipedia/user/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for edits with project, user-type and page-type filter' : '/edits/aggregate/en.wikipedia/group-bot/non-content/daily/2017010100/2017010200'

editsPerPageFixtures

  • 'return 400 for edits per page with all-projects filter' : '/edits/per-page/all-projects/Fake_page/all-editor-types/daily/2017010100/201701000'
  • 'return 400 for edits per page with typo in date' : '/edits/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010100/2017010a00'
  • 'return 400 for edits per page with invalid date (end before start)' : '/edits/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010200/2017010100'
  • 'return 400 for edits per page with too-long span (367 days)' : '/edits/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010100/2018010400'
  • 'return 400 for edits per page with invalid granularity' : '/edits/per-page/en.wikipedia/Fake_page/all-editor-types/wrong/2017010100/2017010200'
  • 'return 400 for edits per page with invalid editor-type' : '/edits/per-page/en.wikipedia/Fake_page/wrong/daily/2017010100/2017010200'
  • 'return 400 for monthly edits per page and no full-month dates' : '/edits/per-page/en.wikipedia/Fake_page/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for edits per page daily with only page-title and project filter' : '/edits/per-page/EN.wikipedia.org/Fake_page/all-editor-types/daily/2017010100/2017010200'
  • 'return 200 and results for edits per page monthly with only page-title and project filter and no hours' : '/edits/per-page/en.wikipedia/Fake_page/all-editor-types/monthly/20170101/20170210'
  • 'return 200 with results for edits per page with project, page-title and editor-type filter' : '/edits/per-page/en.wikipedia/Fake_page/user/daily/2017010100/2017010200'

editsPerEditorFixtures

  • 'return 400 for edits per editor with all-projects filter' : '/edits/per-editor/all-projects/Fake_User/all-page-types/daily/2017010100/201701000'
  • 'return 400 for edits per editor with typo in date' : '/edits/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010100/2017010a00'
  • 'return 400 for edits per editor with invalid date (end before start)' : '/edits/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010200/2017010100'
  • 'return 400 for edits per editor with too long span (367 days)' : '/edits/per-editor/en.wikipedia/Fake_User/all-page-types/monthly/2017010100/2018010400'
  • 'return 400 for edits per editor with invalid granularity' : '/edits/per-editor/en.wikipedia/Fake_User/all-page-types/wrong/2017010100/2017010200'
  • 'return 400 for edits per editor with invalid page-type' : '/edits/per-editor/en.wikipedia/Fake_User/wrong/daily/2017010100/2017010200'
  • 'return 400 for monthly edits per editor and no full-month dates' : '/edits/per-editor/en.wikipedia/Fake_User/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for edits per editor daily with only project and user-text filter' : '/edits/per-editor/EN.wikipedia.org/Fake_User/all-page-types/daily/2017010100/2017010200'
  • 'return 200 and results for edits per editor monthly with only project and user-text filter and no hours' : '/edits/per-editor/en.wikipedia/Fake_User/all-page-types/monthly/20170101/20170210'
  • 'return 200 with results for edits per editor with project, user-text and page-type filter' : '/edits/per-editor/en.wikipedia/Fake_User/content/daily/2017010100/2017010200'

netBytesDiffFixtures

  • 'return 400 for net bytes-difference with typo in date' : '/bytes-difference/net/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010a00'
  • 'return 400 for net bytes-difference with invalid date (end before start)' : '/bytes-difference/net/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010200/2017010100'
  • 'return 400 for net bytes-difference with invalid granularity' : '/bytes-difference/net/aggregate/all-projects/all-editor-types/all-page-types/wrong/2017010100/2017010200'
  • 'return 400 for net bytes-difference with invalid page-type' : '/bytes-difference/net/aggregate/all-projects/all-editor-types/wrong/daily/2017010100/2017010200'
  • 'return 400 for net bytes-difference with invalid user-type' : '/bytes-difference/net/aggregate/all-projects/wrong/all-page-types/daily/2017010100/2017010200'
  • 'return 400 for monthly net bytes-difference and no full-month dates' : '/bytes-difference/net/aggregate/all-projects/all-editor-types/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for net bytes-difference daily without filters' : '/bytes-difference/net/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 and results for net bytes-difference monthly without filters nor hours' : '/bytes-difference/net/aggregate/all-projects/all-editor-types/all-page-types/monthly/20170101/20170210'
  • 'return 200 with results for net bytes-difference with uppercase project filter' : '/bytes-difference/net/aggregate/EN.wikipedia.org/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for net bytes-difference with project and user-type filter' : '/bytes-difference/net/aggregate/en.wikipedia/user/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for net bytes-difference with project, user-type and page-type filter' : '/bytes-difference/net/aggregate/en.wikipedia/group-bot/non-content/daily/2017010100/2017010200'

netBytesDiffPerPageFixtures

  • 'return 400 for net bytes-difference per page with all-projects filter' : '/bytes-difference/net/per-page/all-projects/Fake_page/all-editor-types/daily/2017010100/201701000'
  • 'return 400 for net bytes-difference per page with typo in date' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010100/2017010a00'
  • 'return 400 for net bytes-difference per page with invalid date (end before start)' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010200/2017010100'
  • 'return 400 for net bytes-difference per page with too long span (367 days)' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010100/2018010400'
  • 'return 400 for net bytes-difference per page with invalid granularity' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/all-editor-types/wrong/2017010100/2017010200'
  • 'return 400 for net bytes-difference per page with invalid editor-type' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/wrong/daily/2017010100/2017010200'
  • 'return 400 for monthly net bytes-difference per page and no full-month dates' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for net bytes-difference per page daily with only page-title and project filter' : '/bytes-difference/net/per-page/EN.wikipedia.org/Fake_page/all-editor-types/daily/2017010100/2017010200'
  • 'return 200 and results for net bytes-difference per page monthly with only page-title and project filter and no hours' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/all-editor-types/monthly/20170101/20170210'
  • 'return 200 with results for net bytes-difference per page with project, page-title and editor-type filter' : '/bytes-difference/net/per-page/en.wikipedia/Fake_page/user/daily/2017010100/2017010200'

netBytesDiffPerEditorFixtures

  • 'return 400 for net bytes-difference per editor with all-projects filter' : '/bytes-difference/net/per-editor/all-projects/Fake_User/all-page-types/daily/2017010100/201701000'
  • 'return 400 for net bytes-difference per editor with typo in date' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010100/2017010a00'
  • 'return 400 for net bytes-difference per editor with invalid date (end before start)' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010200/2017010100'
  • 'return 400 for net bytes-difference per editor with too long span (367 days)' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010100/2019010100'
  • 'return 400 for net bytes-difference per editor with invalid granularity' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/all-page-types/wrong/2017010100/2017010200'
  • 'return 400 for net bytes-difference per editor with invalid page-type' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/wrong/daily/2017010100/2017010200'
  • 'return 400 for monthly net bytes-difference per editor and no full-month dates' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for net bytes-difference per editor daily with only project and user-text filter' : '/bytes-difference/net/per-editor/EN.wikipedia.org/Fake_User/all-page-types/daily/2017010100/2017010200'
  • 'return 200 and results for net bytes-difference per editor monthly with only project and user-text filter and no hours' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/all-page-types/monthly/20170101/20170210'
  • 'return 200 with results for net bytes-difference per editor with project, user-text and page-type filter' : '/bytes-difference/net/per-editor/en.wikipedia/Fake_User/content/daily/2017010100/2017010200'

absBytesDiffFixtures

  • 'return 400 for absolute bytes-difference with typo in date' : '/bytes-difference/absolute/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010a00'
  • 'return 400 for absolute bytes-difference with invalid date (end before start)' : '/bytes-difference/absolute/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010200/2017010100'
  • 'return 400 for absolute bytes-difference with invalid granularity' : '/bytes-difference/absolute/aggregate/all-projects/all-editor-types/all-page-types/wrong/2017010100/2017010200'
  • 'return 400 for absolute bytes-difference with invalid page-type' : '/bytes-difference/absolute/aggregate/all-projects/all-editor-types/wrong/daily/2017010100/2017010200'
  • 'return 400 for absolute bytes-difference with invalid user-type' : '/bytes-difference/absolute/aggregate/all-projects/wrong/all-page-types/daily/2017010100/2017010200'
  • 'return 400 for monthly absolute bytes-difference and no full-month dates' : '/bytes-difference/absolute/aggregate/all-projects/all-editor-types/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for absolute bytes-difference daily without filters' : '/bytes-difference/absolute/aggregate/all-projects/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 and results for absolute bytes-difference monthly without filters nor hours' : '/bytes-difference/absolute/aggregate/all-projects/all-editor-types/all-page-types/monthly/20170101/20170210'
  • 'return 200 with results for absolute bytes-difference with uppercase project filter' : '/bytes-difference/absolute/aggregate/EN.wikipedia.org/all-editor-types/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for absolute bytes-difference with project and user-type filter' : '/bytes-difference/absolute/aggregate/en.wikipedia/user/all-page-types/daily/2017010100/2017010200'
  • 'return 200 with results for absolute bytes-difference with project, user-type and page-type filter' : '/bytes-difference/absolute/aggregate/en.wikipedia/group-bot/non-content/daily/2017010100/2017010200'

absBytesDiffPerPageFixtures

  • 'return 400 for absolute bytes-difference per page with all-projects filter' : '/bytes-difference/absolute/per-page/all-projects/Fake_page/all-editor-types/daily/2017010100/201701000'
  • 'return 400 for absolute bytes-difference per page with typo in date' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010100/2017010a00'
  • 'return 400 for absolute bytes-difference per page with invalid date (end before start)' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010200/2017010100'
  • 'return 400 for absolute bytes-difference per page with too long span (367 days)' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/all-editor-types/daily/2017010200/2018030100'
  • 'return 400 for absolute bytes-difference per page with invalid granularity' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/all-editor-types/wrong/2017010100/2017010200'
  • 'return 400 for absolute bytes-difference per page with invalid editor-type' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/wrong/daily/2017010100/2017010200'
  • 'return 400 for monthly absolute bytes-difference per page and no full-month dates' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for absolute bytes-difference per page daily with only page-title and project filter' : '/bytes-difference/absolute/per-page/EN.wikipedia.org/Fake_page/all-editor-types/daily/2017010100/2017010200'
  • 'return 200 and results for absolute bytes-difference per page monthly with only page-title and project filter and no hours' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/all-editor-types/monthly/20170101/20170210'
  • 'return 200 with results for absolute bytes-difference per page with project, page-title and editor-type filter' : '/bytes-difference/absolute/per-page/en.wikipedia/Fake_page/user/daily/2017010100/2017010200'

absBytesDiffPerEditorFixtures

  • 'return 400 for absolute bytes-difference per editor with all-projects filter' : '/bytes-difference/absolute/per-editor/all-projects/Fake_User/all-page-types/daily/2017010100/201701000'
  • 'return 400 for absolute bytes-difference per editor with typo in date' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010100/2017010a00'
  • 'return 400 for absolute bytes-difference per editor with invalid date (end before start)' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010200/2017010100'
  • 'return 400 for absolute bytes-difference per editor with too long span (367 days)' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/all-page-types/daily/2017010200/2019010100'
  • 'return 400 for absolute bytes-difference per editor with invalid granularity' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/all-page-types/wrong/2017010100/2017010200'
  • 'return 400 for absolute bytes-difference per editor with invalid page-type' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/wrong/daily/2017010100/2017010200'
  • 'return 400 for monthly absolute bytes-difference per editor and no full-month dates' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/all-page-types/monthly/2017010100/2017010200'
  • 'return 200 and results for absolute bytes-difference per editor daily with only project and user-text filter' : '/bytes-difference/absolute/per-editor/EN.wikipedia.org/Fake_User/all-page-types/daily/2017010100/2017010200'
  • 'return 200 and results for absolute bytes-difference per editor monthly with only project and user-text filter and no hours' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/all-page-types/monthly/20170101/20170210'
  • 'return 200 with results for absolute bytes-difference per editor with project, user-text and page-type filter' : '/bytes-difference/absolute/per-editor/en.wikipedia/Fake_User/content/daily/2017010100/2017010200'

topEditedPagesPerEditsFixtures

  • 'return 400 for top-edited-pages-per-edits with typo in date' : '/edited-pages/top-by-edits/en.wikipedia/all-editor-types/all-page-types/2017a/01/01'
  • 'return 400 for top-edited-pages-per-edits with invalid date' : '/edited-pages/top-by-edits/en.wikipedia/all-editor-types/all-page-types/2017/21/02'
  • 'return 400 for top-edited-pages-per-edits with invalid page-type' : '/edited-pages/top-by-edits/en.wikipedia/all-editor-types/wrong/2017/01/01'
  • 'return 400 for top-edited-pages-per-edits with invalid user-type' : '/edited-pages/top-by-edits/en.wikipedia/wrong/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-edits daily with all-projects filter' : '/edited-pages/top-by-edits/all-projects/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-edits daily with uppercase and .org project filter' : '/edited-pages/top-by-edits/EN.wikipedia.org/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-edits monthly with only project filter but no hour' : '/edited-pages/top-by-edits/en.wikipedia/all-editor-types/all-page-types/2017/01/all-days'
  • 'return 200 with results for top-edited-pages-per-edits with project and editor-type filter' : '/edited-pages/top-by-edits/en.wikipedia/anonymous/all-page-types/2017/01/01'
  • 'return 200 with results for top-edited-pages-per-edits with project, editor-type and page-type filter' : '/edited-pages/top-by-edits/en.wikipedia/user/non-content/2017/01/01'

topEditedPagesPerNetBytesDiffFixtures

  • 'return 400 for top-edited-pages-per-net-bytes-diff with typo in date' : '/edited-pages/top-by-net-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/01/0a'
  • 'return 400 for top-edited-pages-per-net-bytes-diff with invalid date' : '/edited-pages/top-by-net-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/21/02'
  • 'return 400 for top-edited-pages-per-net-bytes-diff with invalid page-type' : '/edited-pages/top-by-net-bytes-difference/en.wikipedia/all-editor-types/wrong/2017/01/01'
  • 'return 400 for top-edited-pages-per-net-bytes-diff with invalid user-type' : '/edited-pages/top-by-net-bytes-difference/en.wikipedia/wrong/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-net-bytes-diff daily with all-projects filter' : '/edited-pages/top-by-net-bytes-difference/all-projects/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-net-bytes-diff daily with uppercase and .org project filter' : '/edited-pages/top-by-net-bytes-difference/EN.wikipedia.org/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-net-bytes-diff monthly with only project filter but no hour' : '/edited-pages/top-by-net-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/01/all-days'
  • 'return 200 with results for top-edited-pages-per-net-bytes-diff with project and editor-type filter' : '/edited-pages/top-by-net-bytes-difference/en.wikipedia/anonymous/all-page-types/2017/01/01'
  • 'return 200 with results for top-edited-pages-per-net-bytes-diff with project, editor-type and page-type filter' : '/edited-pages/top-by-net-bytes-difference/en.wikipedia/user/non-content/2017/01/01'

topEditedPagesPerAbsBytesDiffFixtures

  • 'return 400 for top-edited-pages-per-absolute-bytes-diff with typo in date' : '/edited-pages/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/0a/01'
  • 'return 400 for top-edited-pages-per-absolute-bytes-diff with invalid date (end before start)' : '/edited-pages/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/01/45'
  • 'return 400 for top-edited-pages-per-absolute-bytes-diff with invalid page-type' : '/edited-pages/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/wrong/2017/01/01'
  • 'return 400 for top-edited-pages-per-absolute-bytes-diff with invalid user-type' : '/edited-pages/top-by-absolute-bytes-difference/en.wikipedia/wrong/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-absolute-bytes-diff daily with all-projects filter' : '/edited-pages/top-by-absolute-bytes-difference/all-projects/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-absolute-bytes-diff daily with uppercase and .org project filter' : '/edited-pages/top-by-absolute-bytes-difference/EN.wikipedia.org/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-edited-pages-per-absolute-bytes-diff monthly with only project filter but no hour' : '/edited-pages/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/01/all-days'
  • 'return 200 with results for top-edited-pages-per-absolute-bytes-diff with project and editor-type filter' : '/edited-pages/top-by-absolute-bytes-difference/en.wikipedia/anonymous/all-page-types/2017/01/01'
  • 'return 200 with results for top-edited-pages-per-absolute-bytes-diff with project, editor-type and page-type filter' : '/edited-pages/top-by-absolute-bytes-difference/en.wikipedia/user/non-content/2017/01/01'

topEditorsPerEditsFixtures

  • 'return 400 for top-editors-per-edits with typo in date' : '/editors/top-by-edits/en.wikipedia/all-editor-types/all-page-types/201a/01/01'
  • 'return 400 for top-editors-per-edits with invalid date' : '/editors/top-by-edits/en.wikipedia/all-editor-types/all-page-types/2017/41/02'
  • 'return 400 for top-editors-per-edits with invalid page-type' : '/editors/top-by-edits/en.wikipedia/all-editor-types/wrong/2017/01/01'
  • 'return 400 for top-editors-per-edits with invalid user-type' : '/editors/top-by-edits/en.wikipedia/wrong/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-edits daily with all-projects filter' : '/editors/top-by-edits/all-projects/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-edits daily with uppercase and .org project filter' : '/editors/top-by-edits/EN.wikipedia.org/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-edits monthly with only project filter but no hour' : '/editors/top-by-edits/en.wikipedia/all-editor-types/all-page-types/2017/01/all-days'
  • 'return 200 with results for top-editors-per-edits with project and editor-type filter' : '/editors/top-by-edits/en.wikipedia/anonymous/all-page-types/2017/01/01'
  • 'return 200 with results for top-editors-per-edits with project, editor-type and page-type filter' : '/editors/top-by-edits/en.wikipedia/user/non-content/2017/01/01'

topEditorsPerNetBytesDiffFixtures

  • 'return 400 for top-editors-per-net-bytes-diff with typo in date' : '/editors/top-by-net-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/a1/01'
  • 'return 400 for top-editors-per-net-bytes-diff with invalid date' : '/editors/top-by-net-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/02/31'
  • 'return 400 for top-editors-per-net-bytes-diff with invalid page-type' : '/editors/top-by-net-bytes-difference/en.wikipedia/all-editor-types/wrong/2017/01/01'
  • 'return 400 for top-editors-per-net-bytes-diff with invalid user-type' : '/editors/top-by-net-bytes-difference/en.wikipedia/wrong/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-net-bytes-diff daily with all-projects filter' : '/editors/top-by-net-bytes-difference/all-projects/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-net-bytes-diff daily with uppercase and .org project filter' : '/editors/top-by-net-bytes-difference/EN.wikipedia.org/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-net-bytes-diff monthly with only project filter but no hour' : '/editors/top-by-net-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/01/all-days'
  • 'return 200 with results for top-editors-per-net-bytes-diff with project and editor-type filter' : '/editors/top-by-net-bytes-difference/en.wikipedia/anonymous/all-page-types/2017/01/01'
  • 'return 200 with results for top-editors-per-net-bytes-diff with project, editor-type and page-type filter' : '/editors/top-by-net-bytes-difference/en.wikipedia/user/non-content/2017/01/01'

topEditorsPerAbsBytesDiffFixtures

  • 'return 400 for top-editors-per-absolute-bytes-diff with typo in date' : '/editors/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/01/a1'
  • 'return 400 for top-editors-per-absolute-bytes-diff with invalid date' : '/editors/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/13/01'
  • 'return 400 for top-editors-per-absolute-bytes-diff with invalid page-type' : '/editors/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/wrong/2017/01/01'
  • 'return 400 for top-editors-per-absolute-bytes-diff with invalid user-type' : '/editors/top-by-absolute-bytes-difference/en.wikipedia/wrong/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-absolute-bytes-diff daily with all-projects filter' : '/editors/top-by-absolute-bytes-difference/all-projects/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-absolute-bytes-diff daily with uppercase and .org project filter' : '/editors/top-by-absolute-bytes-difference/EN.wikipedia.org/all-editor-types/all-page-types/2017/01/01'
  • 'return 200 and results for top-editors-per-absolute-bytes-diff monthly with only project filter but no hour' : '/editors/top-by-absolute-bytes-difference/en.wikipedia/all-editor-types/all-page-types/2017/01/all-days'
  • 'return 200 with results for top-editors-per-absolute-bytes-diff with project and editor-type filter' : '/editors/top-by-absolute-bytes-difference/en.wikipedia/anonymous/all-page-types/2017/01/01'
  • 'return 200 with results for top-editors-per-absolute-bytes-diff with project, editor-type and page-type filter' : '/editors/top-by-absolute-bytes-difference/en.wikipedia/user/non-content/2017/01/01'

If that seems like a lot of tests, that's because it is. I count 196. The current fixtures.js uses helper functions to avoid cut-and-paste boilerplate. While the AQS 2.0 test implementation will be different, we should still consider a similar pattern of helper functions to enhance readability.

Test data required to support the above tests

  • at least two wikipedias, data for a select set of pages (multiple wikipedias are needed for "return 200 with results for new-pages with project-family filter".
  • some new pages (at least two created on different days of the same month)
  • edited pages that fulfill 'return 200 with results for edited-pages with project and editor-type filter' : '/edited-pages/aggregate/en.wikipedia/anonymous/all-page-types/all-activity-levels/daily/2017010100/2017010200'
  • edited pages that fulfill 'return 200 with results for edited-pages with project, editor-type and page-type filter' : '/edited-pages/aggregate/en.wikipedia/user/non-content/all-activity-levels/daily/2017010100/2017010200'
  • edited pages that fulfill 'return 200 with results for edited-pages with project, editor-type, page-type and activity-level filter' : '/edited-pages/aggregate/en.wikipedia/group-bot/content/25..99-edits/daily/2017010100/2017010200'
  • newly registered users (at least two on different days within the time frame, one on en.wikipedia.org and one on any wiktionary)
  • multiple edits by the same editor on different days in the same month on enwiki
  • edits and new pages should include both content and non-content pages (non-content pages are pages in non-content namespaces)

Note that we do not need to test against the same dates or pages as the existing production service. Our tests just need to cover the same types of functionality.

This is all pretty common data. Should be able to get it by querying data for a few days on:

  • en.wikipedia.org
  • some other wikipedia
  • en.wiktionary.org

We'll need to make sure this includes all the various types of interactions needed (bot users, newly registered users, multiple edits by the same user, etc.) It is probably possible to grab an overabundance of data, execute the queries for the tests, see what data corresponds to them, and either requery more specific data or filter out irrelevant data (whichever seems easier). Then include only that data in the uploaded image. This means only the person who grabs and cleans the test data will have to deal with a way-too-large dataset. Testers will only get the reduced/cleaned data, thereby managing image size.

Related Objects
Search...

Event Timeline

BPirkle created this task.Sep 1 2022, 3:06 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 1 2022, 3:06 AM
BPirkle added a parent task: T263489: AQS 2.0.Sep 1 2022, 3:06 AM
BPirkle added a subscriber: Unknown Object (User).Sep 7 2022, 1:58 PM
BPirkle added a subscriber: SGupta-WMF.Sep 8 2022, 4:58 PM
BPirkle updated the task description. (Show Details)Sep 11 2022, 4:23 PM
BPirkle updated the task description. (Show Details)Sep 11 2022, 11:43 PM
VirginiaPoundstone moved this task from Incoming to Needs Grooming on the API Platform board.Sep 13 2022, 10:20 PM
BPirkle mentioned this in T317730: AQS 2.0: Wikistats2: Implement Unit Tests.Sep 14 2022, 3:34 AM
VirginiaPoundstone moved this task from Needs Grooming to Sprint Backlog on the API Platform board.Sep 14 2022, 4:04 PM
BPirkle added a parent task: T311190: Establish testing procedure for Druid-based endpoints.Sep 14 2022, 7:30 PM
BPirkle removed a parent task: T263489: AQS 2.0.
BPirkle mentioned this in T317803: AQS 2.0: Extract production testing data for Druid-based endpoints.Sep 14 2022, 7:40 PM
BPirkle moved this task from Sprint Backlog to Backlog on the API Platform board.Sep 14 2022, 7:47 PM
BPirkle moved this task from Backlog to QA/Review on the API Platform board.Sep 15 2022, 10:27 PM
EChukwukere-WMF subscribed.Sep 29 2022, 2:24 AM

Upon discussion with @BPirkle I have commenced building up of this tests. I will be using a python-cucumber framework for the tests.

I'd likek to mention that there are weeks worth of work here and it will be best to break these down further probably by service endpoints.

The tests will live in this repo I created with the help of Bill : https://gitlab.wikimedia.org/repos/generated-data-platform/aqs/aqs_tests

I am open to any questions and/or suggestions

BPirkle added a comment.Sep 29 2022, 2:41 AM

@EChukwukere-WMF , the above may not be the best set of tests to start with. They correspond to endpoints that we have not yet implemented, and probably won't finish this sprint. In other words, the endpoints corresponding to the above tests don't exist yet, and probably won't for a couple of weeks.

The tests for the Pageviews service, listed in T299735: AQS 2.0: Pageviews: Implement Integration Tests might be a better choice. That service is essentially complete.

With that said, our current plan is for all the tests in both those lists (this task and T299735) to be implemented as unit tests within the services. Those tests will execute against controlled testing environments (one for Cassandra, another for Druid).

I feel that it would also be useful to implement true integration tests like you're talking about, that test against production data, and which can be executed against both the existing production service that is already in place, and against the new AQS 2.0 services, once they are in production but before they are exposed to end users. The idea is to ensure compatibility, so that when we route actual traffic to AQS 2.0, we're less likely to break existing callers.

I'm not sure that the integration tests need to duplicate the unit tests on a test-for-test basis. It might be that a more limited test suite would be sufficient to ensure compatibility. It might also be that you would want to design compatibility tests (which should be very particular about small details of things like response formats) differently than the unit tests (which are mostly to catch bugs).

My point is, I'd hate for you to spend weeks of work when it isn't necessary. I'd suggest that you write a couple of small-ish proof-of-concept tests for the pageviews service, using the python-cucumber framework, and confirm that you can execute them locally against a local copy of that service. That'll give us a little time to figure out exactly how to approach integration testing (and hopefully save you a lot of time!)

Feel free to either drop in on one of the Code Mob meetings, or else set up a meeting with myself and maybe @VirginiaPoundstone , and we can work out the details.

EChukwukere-WMF added a subscriber: Atieno.Sep 29 2022, 5:16 PM

@BPirkle I clearly see and understand what you are saying. So thanks for clarifying all that. one last thing, the pageviews test you are referring to are the ones we talked about in our last meeting here : https://wikimedia.org/api/rest_v1/#/Pageviews%20data

as those are the ones I am now working with. Please confirm so as soon I am done I will send them over or do a demo when I get it up and running

cc @Atieno

BPirkle added a comment.Sep 29 2022, 10:20 PM
In T316849#8273336, @EChukwukere-WMF wrote:

@BPirkle I clearly see and understand what you are saying. So thanks for clarifying all that. one last thing, the pageviews test you are referring to are the ones we talked about in our last meeting here : https://wikimedia.org/api/rest_v1/#/Pageviews%20data

as those are the ones I am now working with. Please confirm so as soon I am done I will send them over or do a demo when I get it up and running

Confirmed, those are the endpoints I am suggesting you focus on first.

VirginiaPoundstone added a parent task: T317790: <Spike> AQS 2.0 Testing Plan.Oct 7 2022, 7:20 PM
VirginiaPoundstone moved this task from QA/Review to Should do next on the API Platform board.Oct 26 2022, 1:19 PM
VirginiaPoundstone added a project: AQS2.0.Nov 14 2022, 7:28 PM
VirginiaPoundstone moved this task from Incoming to In Current Sprint on the AQS2.0 board.Nov 14 2022, 7:41 PM
BPirkle mentioned this in T317731: AQS 2.0: Wikistats2: Implement Test Framework.Jan 25 2023, 1:00 AM
BPirkle mentioned this in T327835: AQS 2.0: Implement Test Framework for Druid-based Endpoints.
BPirkle mentioned this in T327836: AQS 2.0: Edit Analytics: Implement Unit Tests.Jan 25 2023, 1:05 AM
BPirkle mentioned this in T327837: AQS 2.0: Editor Analytics: Implement Unit Tests.
JArguello-WMF closed this task as Resolved.Feb 1 2023, 7:30 PM
BPirkle mentioned this in T328969: AQS 2.0: Revisit in-service testing approach.Feb 6 2023, 9:06 PM
BPirkle mentioned this in T334851: Define a procedure/pattern to populate test environments.Apr 17 2023, 3:11 PM
BPirkle mentioned this in T336382: Edit Analytics: Create Druid test data environment.May 10 2023, 1:42 PM
Sfaci mentioned this in T340107: AQS 2.0: Implement Test Framework for Druid-based Endpoints: Editor Analytics.Jun 22 2023, 12:14 PM
Sfaci mentioned this in T340108: AQS 2.0: Implement Test Framework for Druid-based Endpoints: Edit Analytics.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits