Page MenuHomePhabricator
Create Task
Maniphest T273554

Make config change to enable Reply Tool A/B test
Closed, ResolvedPublic
Actions

Description

This task is about enabling/turning on the Reply Tool A/B test at the 22 21 ===Participating wikis (listed below) where the A/B has NOT yet been deployed.[i]

Deployment timing

We would like for the A/B test to be made available at the ===Participating wikis on THURSDAY, 11-FEBRUARY-2021.

Participating wikis

NumberWikiCodeStatus
1.French Wikipediafrwiki
2.Spanish Wikipediaeswiki
3.Italian Wikipediaitwiki
4.Japanese Wikipediajawiki
5.Persian Wikipediafawiki
6.Polish Wikipediaplwiki
7.Hebrew Wikipediahewiki
8.Dutch Wikipedianlwiki
9.Hindi Wikipediahiwiki
10.Korean Wikipediakowiki
11.Vietnamese Wikipediaviwiki
12.Thai Wikipediathwiki
13.Portuguese Wikipediaptwiki
14.Bengali Wikipediabnwiki
15.Egyptian Wikipediaarzwiki
16.Swahili Wikipediaswwiki
17.Chinese Wikipediazhwiki
18.Ukrainian Wikipediaukwiki
19.Indonesia WikipediaidwikiEnabled in T268191#6767541
20.Amharic Wikipediaamwiki
21.Oromo Wikipediaomwiki
22.Afrikaans Wikipediaafwiki

Testing instructions

  1. At even-numbered Wikipedias listed in the Participating wikis section above, do the following:
  2. Post a test comment using the Reply Tool (source or visual mode is fine) to a page of your choosing (e.g. your Sandbox talk page) and verify the comment is being posted to the page as you expect it to.
  3. Visit a random talk page and verify [ reply ] links are appearing as you expect them to
  4. Click a [ reply ] link and verify the Reply Tool opens as you expect it to.
  5. Document (e.g. file a ticket and/or comment here if you notice any unexpected behavior as part of completing steps "2, 3 and 4.)

Done

  • @ppelberg to comment on T273406 once the ===Deployment timing has been set
  • The Reply Tool A/B test is made available at the ===Participating wikis on the date specified in the === Deployment timing section

i. The A/B test was enabled at id.wiki in T268191#6767541.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Enable DiscussionTools Reply Tool A/B testoperations/mediawiki-configmaster+21 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

ppelberg created this task.Feb 1 2021, 11:05 PM
Restricted Application added subscribers: Huji, Stang, Base, revi. · View Herald TranscriptFeb 1 2021, 11:05 PM
ppelberg moved this task from Inbox to Blocked / Needs More Work on the Editing-team (Kanban Board) board.Feb 1 2021, 11:05 PM
ppelberg renamed this task from Make config change to enabled Reply Tool A/B test to Make config change to enable Reply Tool A/B test .Feb 1 2021, 11:23 PM
ppelberg mentioned this in T273406: Announce start of A/B test.Feb 1 2021, 11:30 PM
ppelberg added a subscriber: DLynch.

Deployment timing

Per the conversation @DLynch, @MNeisler and I had in chat today, we would like for the A/B test to begin this Thursday, 4-February-2021.

I've updated the task description to reflect the above.

ppelberg triaged this task as High priority.Feb 1 2021, 11:31 PM
ppelberg moved this task from Blocked / Needs More Work to Ready to Be Worked On on the Editing-team (Kanban Board) board.
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)Feb 1 2021, 11:34 PM
Shizhao added a project: Chinese-Sites.Feb 2 2021, 3:38 AM
Shizhao moved this task from Backlog to Research on the Chinese-Sites board.
Amorymeltzer unsubscribed.Feb 2 2021, 6:25 PM
matmarex subscribed.Feb 2 2021, 6:25 PM
ppelberg added a comment.Feb 2 2021, 9:37 PM

Deployment timing update
Per today's standup, the A/B test should NOT be enabled until after wmf/1.36.0-wmf.29 reaches all Wikipedias.

As of 21:35 today, 2-Feb-2021, wmf/1.36.0-wmf.29 remains blocked. More details in T271343.

I've updated the task description to reflect this.

ppelberg updated the task description. (Show Details)Feb 2 2021, 9:38 PM
gerritbot added a comment.Feb 3 2021, 2:28 PM

Change 661373 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[operations/mediawiki-config@master] Enable DiscussionTools Reply Tool A/B test

https://gerrit.wikimedia.org/r/661373

gerritbot added a project: Patch-For-Review.Feb 3 2021, 2:28 PM
matmarex added a comment.Feb 4 2021, 7:34 PM
In T273554#6797811, @ppelberg wrote:

Deployment timing update
Per today's standup, the A/B test should NOT be enabled until after wmf/1.36.0-wmf.29 reaches all Wikipedias.

As of 21:35 today, 2-Feb-2021, wmf/1.36.0-wmf.29 remains blocked. More details in T271343.

Looks like this will have to wait until next week.

ppelberg added a comment.EditedFeb 8 2021, 7:02 PM

Next steps

  • As discussed during today's team standup, @DLynch is investigating what would be involved with backporting all of the necessary patches to start the A/B test without needing to depend on the train running, which is currently blocked (see: T271343).
DLynch added a comment.Feb 8 2021, 7:41 PM

In a backport window we would need to put:

  1. The config patch Bartosz wrote above
  2. The logging patch for VisualEditor for DT buckets
  3. The logging patch for WikiEditor for DT buckets

They're pretty easy to cherry-pick onto the current state of the branch.

In fact, the only likely conflicts are with these, which are not part of this... but if we get into anything for the new discussions feature we'd probably also want, related to T272544:

  1. The logging patch for VisualEditor for proper new-section tracking
  2. The logging patch for WikiEditor for proper new-section tracking
  3. The schema-bump patch to WikimediaEvents to support the previous patches
ppelberg added a subscriber: • Whatamidoing-WMF.Feb 9 2021, 10:31 PM

Deployment timing
The deployment is scheduled for today, 9-Feb-2021, at 4pm PST / 12am UTC on 10-Feb-2021.

cc @Whatamidoing-WMF

ppelberg updated the task description. (Show Details)Feb 9 2021, 10:31 PM
ppelberg updated the task description. (Show Details)
ppelberg added a comment.Feb 10 2021, 2:58 AM
In T273554#6816841, @ppelberg wrote:

Deployment timing
The deployment is scheduled for today, 9-Feb-2021, at 4pm PST / 12am UTC on 10-Feb-2021.

cc @Whatamidoing-WMF

This deployment was blocked and accordingly, did not happen. We'll revisit this tomorrow (10-February).

ppelberg added a comment.Feb 10 2021, 6:31 PM

Next steps

  • Today:
    • Do nothing and observe whether 1.36.0-wmf.30 is rolled out to Group 1
  • Tomorrow:
ppelberg added a comment.Feb 11 2021, 6:54 PM
In T273554#6819662, @ppelberg wrote:

Next steps

Considering 1.36.0-wmf.30 has been rolled out to Group 1 [i], we're going to move forward with the following: deploy https://gerrit.wikimedia.org/r/661373 during today's, 4pm PST / 00:00 UTC backport window. [ii]

i. https://versions.toolforge.org/
ii. https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20210211T2200

ppelberg updated the task description. (Show Details)Feb 11 2021, 6:55 PM
gerritbot added a comment.Feb 12 2021, 12:56 AM

Change 661373 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable DiscussionTools Reply Tool A/B test

https://gerrit.wikimedia.org/r/661373

Stashbot added a comment.Feb 12 2021, 1:04 AM

Mentioned in SAL (#wikimedia-operations) [2021-02-12T01:04:12Z] <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 389f7f1fdc9ad4a0c163ccfe1d80f2aaec7f8038: Enable DiscussionTools Reply Tool A/B test (T273554) (duration: 01m 08s)

Maintenance_bot removed a project: Patch-For-Review.Feb 12 2021, 1:11 AM
ppelberg mentioned this in T252057: Run A/B test to evaluate impact of Reply tool .Feb 12 2021, 2:52 AM
matmarex moved this task from Ready to Be Worked On to Ready for Sign Off on the Editing-team (Kanban Board) board.Feb 12 2021, 3:20 PM
ppelberg moved this task from Ready for Sign Off to QA on the Editing-team (Kanban Board) board.Feb 13 2021, 12:27 AM
ppelberg added a project: Editing QA.
ppelberg moved this task from Inbox to High Priority on the Editing QA board.
ppelberg updated the task description. (Show Details)Feb 25 2021, 8:19 PM
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)Feb 25 2021, 8:22 PM
ppelberg updated the task description. (Show Details)Mar 1 2021, 8:06 PM
Ryasmeen subscribed.Mar 1 2021, 11:22 PM

@ppelberg, checked the test scenarios on the mentioned 11 wikis. Test results are recorded on this workbook:

https://docs.google.com/spreadsheets/d/1ccTSx-v1yb8kvli7e8djFexXL7jThPHE1WomtkCLG6I/edit#gid=523212285

Found two instances where the Reply tool is not opening when I click the reply links. The pages are:

https://sw.wikipedia.org/wiki/Majadiliano:Mwanzo
https://w.wiki/33MX

ppelberg added a comment.Mar 3 2021, 2:54 AM
In T273554#6873165, @Ryasmeen wrote:

@ppelberg, checked the test scenarios on the mentioned 11 wikis. Test results are recorded on this workbook...

Great – thank you, Rummana.

Found two instances where the Reply tool is not opening when I click the reply links. The pages are:

https://sw.wikipedia.org/wiki/Majadiliano:Mwanzo

Is there a particular comment on the page you could not use the Reply Tool to respond to?

I have not yet found a [ reply ] link on the page that, when clicked, did not cause the reply tool to open. [i]

https://w.wiki/33MX

Do you recall what [ reply ] link(s) you were not able to use to open the Reply Tool?

To me, it looks like most of the [ reply ] links are working as expected. I should note, [ reply ] links within the section titled Іконки соцмереж do NOT open. Although, this seems expected to me considering the section is wrapped in the {{closed}} template which seems to fall under the scenario described here: Wrapper templates.

i. https://sw.wikipedia.org/wiki/Majadiliano:Mwanzoii. https://w.wiki/33MX
Screen Shot 2021-03-02 at 6.38.47 PM.png (1×1 px, 414 KB)
Screen Shot 2021-03-02 at 6.45.50 PM.png (1×1 px, 687 KB)
Ryasmeen added a comment.Mar 3 2021, 4:49 PM
In T273554#6877302, @ppelberg wrote:
In T273554#6873165, @Ryasmeen wrote:

@ppelberg, checked the test scenarios on the mentioned 11 wikis. Test results are recorded on this workbook...

Great – thank you, Rummana.

Found two instances where the Reply tool is not opening when I click the reply links. The pages are:

https://sw.wikipedia.org/wiki/Majadiliano:Mwanzo

Is there a particular comment on the page you could not use the Reply Tool to respond to?

I have not yet found a [ reply ] link on the page that, when clicked, did not cause the reply tool to open. [i]

https://w.wiki/33MX

Do you recall what [ reply ] link(s) you were not able to use to open the Reply Tool?

To me, it looks like most of the [ reply ] links are working as expected. I should note, [ reply ] links within the section titled Іконки соцмереж do NOT open. Although, this seems expected to me considering the section is wrapped in the {{closed}} template which seems to fall under the scenario described here: Wrapper templates.

i. https://sw.wikipedia.org/wiki/Majadiliano:Mwanzoii. https://w.wiki/33MX
Screen Shot 2021-03-02 at 6.38.47 PM.png (1×1 px, 414 KB)
Screen Shot 2021-03-02 at 6.45.50 PM.png (1×1 px, 687 KB)

@ppelberg: For me, none of the Reply links on those two pages are working, I was testing using Chrome 88.0.4324.192.

ppelberg moved this task from QA to Blocked / Needs More Work on the Editing-team (Kanban Board) board.Mar 3 2021, 4:50 PM
ppelberg assigned this task to matmarex.Mar 3 2021, 5:10 PM
matmarex mentioned this in T276393: Reply widget opens for users who can't edit the page.EditedMar 3 2021, 8:48 PM

I can reproduce on these pages while not logged in – it looks like this happens because the pages are protected (so I can't edit them while not logged, and you were probably logged in but not autoconfirmed yet). But in this case we just should not show the links. I filed a separate task: T276393

matmarex moved this task from Blocked / Needs More Work to Ready for Sign Off on the Editing-team (Kanban Board) board.Mar 3 2021, 8:50 PM
ppelberg closed this task as Resolved.Mar 3 2021, 8:52 PM
In T273554#6880635, @matmarex wrote:

I can reproduce on these pages while not logged in – it looks like this happens because the pages are protected (so I can't edit them while not logged, and you were probably logged in but not autoconfirmed yet). But in this case we just should not show the links. I filed a separate task: T276393

Excellent; I'm glad you got to the bottom of this, @matmarex and great job spotting this issue, @Ryasmeen.

We'll verify the fix in T276393.

Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptMar 3 2021, 8:52 PM
Shizhao moved this task from Research to Closed on the Chinese-Sites board.Mar 4 2021, 9:38 AM
ppelberg mentioned this in T276967: Turn off Reply Tool A/B test.Mar 9 2021, 5:29 PM
Stang unsubscribed.Nov 13 2021, 8:57 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits